The Phablet Era - <AUTHOR> http://avc.com/2016/01/the-phablet-era <comment><USER2> A few problems with the predictions...

1\. The first iOS "phablet" only happened in 2014, and was very late to market. This explains why phablet growth in 2014-15 is explosive, but does little to predict future growth.

2\. We are now entering the second year of no significant new small phones being released for either iOS or Android. Of course small phones would fare badly when nothing good is for sale!

The best-selling iPod was the Nano, by a very large margin. The best-selling MacBooks were the 11" Air. If there is any truth to a "6C" coming out this year I expect it will do extremely well. (Unfortunately the days of the perfect 3.5" phones are likely behind us...)

<comment><USER3> > Of course small phones would fare badly when nothing good is for sale!

It sounds like you're treating the introduction of new phone models as an exogenous event, but that's crazy. That's like saying "Of course people don't ride horse-and-buggies any more; hardly anybody make them! That doesn't mean cars are the future."

If manufacturers are introducing fewer models, it's because there is reduced demand for them.

<comment><USER2> You need some period of overlap in order to measure reduced demand. If every horse dropped dead the day cars went on sale, that says nothing about reduced demand for horses.

Thank you for introducing me to the word "exogenous"

<comment><USER4> The samsung note was released in 2011. I'm not sure what the average upgrade cycle is at the moment, but IMHO that seems like sufficient overlap that owners of the original note would have returned to smaller phones if they preferred them. They don't.

<comment><USER5> Trying to switch back to a smaller screen is tough, even if you didn't want a big phone to begin with.

Often I don't want to have a huge phone with me at all, but then I don't have a choice. The long term may be some sort of personal communicator (watch or otherwise.) Small screen, advanced input when necessary by voice. When people want to read things, write messages, watch video, play games, etc, then they can pick up their phablet -- which is effectively a phone sized tablet that fits in your pocket.

The Apple Watch probably is the step towards this direction. However, because it needs the iPhone it misses the full use case, for the moment.

</comment></comment></comment></comment></comment><comment><USER6> > It's very much a normal distribution centered around the 5.5″-6″ mobile > phone (phablet)

o.O This term is so abused...'normal distribution'. My boss would use it all the time without even knowing the parameters.

It ain't normal distribution. This is categorical data which in this case can be fitted with a multinomial distribution.

<comment><USER7> Even worse, the numbers are _differences_ , which can take on negative numbers as well, so it doesn't even make sense to talk about a distribution (in the probability sense).

I read the AVC blog once in a while as I feel he's a pretty well-informed investor, but I often get the feeling he doesn't understand the most basic mathematical/technical concepts. I guess he has a "numbers guy" for that stuff...

</comment></comment><comment><USER8> the smartphone market is crap. it seems almost like the economy of a communist country - everything is the same.

recently my husband and me were buying new phones. i had trouble finding a decent smartphone (decent = good specs) which is <= 4,5". incredibly annoying. i simply want a strong device that fits nicely in my pants pocket, and it seems that's a weird request... my husband wanted one with a physical keyboard - now that's science fiction. completely unavailable in Europe at the time (aside from ancient phones on eBay).

so, this trend analysis seems really ridiculous. consumers have a very limited choice in the matter. the companies behave like sheep, and most writing about the market is sycophantic crap.

<comment><USER9> We seem to be in the tail end of a consolidation phase right now, the one the article is about. I think it's largely brought on by the final commodification of the internals of a phone/tablet; even a $50 Amazon Fire tablet is now a decent tablet, something that wasn't true in that price range two years ago and perhaps even one (I wasn't looking then so I can't quite vouch for it). (Yes, it's probably subsidized a bit, but it still ends up setting the bar.)

Once that completes, I'd expect the market to expand a bit more as companies seek ways to stand out other than "we have the cheapest thing".

But you may never again see a "phone" with a physical keyboard. Too niche. Instead, you'll have to look for a case with integrated keyboard or something: [http://www.amazon.com/Sliding-out-Wireless-Bluetooth- keyboar...](http://www.amazon.com/Sliding-out-Wireless-Bluetooth-keyboard- backlight/dp/B00NOM34WW/ref=sr_1_1?ie=UTF8&qid=1452087874&sr=8-1&keywords=iphone+keyboard+case) I don't think it'll ever be economical again to produce a phone with a keyboard built in, but that niche can be filled with orders-of-magnitude- smaller runs of keyboard cases.

<comment><USER8> the case with a keyboard is an interesting suggestion, thank you!

<comment><USER10> BlackBerry Priv has a keyboard.

</comment></comment></comment></comment><comment><USER11> I believe the increase in "phablet" users is also thanks to emerging markets getting connected.

While living in South America the amount of "big phones" really surprised me, as they seemed so impractical. When I realised after a while that many people used their phone as their only Internet device, it made a lot more sense.

<comment><USER12> Users in Hong Kong (which has great internet access and where presumably the phone is _not_ the only internet devide) appear to like large phones too. It seems counterintuitive but these phones were mostly being used two-handed, and their users had jackets/man-bags/handbags. I wonder if the reason is that users who use Chinese IMEs find the larger screen more convenient to write and view a decent amount of data at the same time.

</comment></comment><comment><USER13> My iPhone 6S is too big for my hand, so I dislike the size of the hardware, but there's no denying that the keyboard's a bit less fiddly and the screen real estate is nice.

It's a compromise, and not one which I really like. I can't see much of an alternative though.

<comment><USER14> The alternative is multiple devices that are cheap and work together seamlessly (I guess the first step is phone companies offering data only plans).

<comment><USER15> Every major carrier offers a data-only plan; they're typically used for tablets, although Samsung made a connected camera and LG made a connected watch (that was pulled off the market after less than a week).

<comment><USER14> Yeah I guess I should have expanded that some, separate data and messaging/phone number have to become popular more than they have to be offered.

I took some note of the LG watch and commented about it here, at which point someone pointed out that the Samsung Gear S has been available since 2014. I guess the tech to jam a cell phone in a watch will be mass market ready in 1 or 2 more generations (the Gear S is _huge_ ). Then the price point just needs to come down a bit.

</comment></comment></comment></comment><comment><USER16> Phablet is such an ugly portmanteau. Let's stick to calling them "huge-ass phones".

<comment><USER17> [https://xkcd.com/37/](https://xkcd.com/37/)

scnr

<comment><USER18> That's still a good description of these devices.

</comment></comment></comment><comment><USER19> Although I can't use my Nexus 6P (5.7") with one hand and it barely fits in my pocket, I love the screen real estate.

Actually, I've not used my laptop or iPad once since I got it a month ago. The mobile-first movement makes a lot of sense, and I can really appreciate how people in third-world countries use their phone as their only computer. The larger screen makes the whole experience significantly better, to the point I don't really miss my tablet or laptop. When I want a larger screen, which is frequently, I use the Chromecast.

<comment><USER20> In Indian colleges we're even encourage to use our phone in class for computer science. Android has got quite a few apps for programming languages.

</comment></comment><comment><USER21> > It's very much a normal distribution centered around the 5.5″-6″ mobile > phone (phablet)

Is that because that's nearly all you can buy in the mid-high end these days?

</comment><comment><USER22> To counterweight defenders of small phones. My wife, her mom, and my mom switched to the 6 plus (from 5s) the minute their contract was up. They didn't even ask about smaller phones. For a big chunk of the population (people who carry purses) the upper limit on cell phone size is not bounded by pocketability. That probably has a significant impact on the market.

<comment><USER23> There's also the chunk of the population with vision problems like presbyopia.

I just upgraded my Moto G to a Moto X Play, and the pocketability hasn't been adversely impacted. My new, bigger, phone doesn't really feel bigger in the pocket (thanks to thinness and smallish bezels), and I can't really tell the difference.

The best thing is that I can set the font size to Huge and read the screen with only a little squinting without having to pull out my reading glasses. I don't think there's any way I could go back to a screen the size of an iPhone 4/5.

<comment><USER24> There's also those of us who are just bigger people. From fingertip to palm my hands are on the order of 10 inches long. The standard phones of a few years ago were incredibly difficult for me to use. I still have an iPhone 4S that I use in a speaker dock for media and I can easily (and regularly) press 4 keys at once with one touch.

That said, there's clearly a market for smaller phones still, just not from me, and I don't understand why the manufacturers don't see this.

</comment></comment></comment><comment><USER25> It makes me a little sad that simply making screens bigger (for whatever purpose) is considered technological progress.

Most of the high-end phones now seem to settle around the 5+ inch margin, leaving those of us who need a good smaller phone with little to no choice.

Even with good build quality and lighter weight, I would take a phone I could physically hold in one hand over something which needs software gimmicks (Samsung) for one-hand use. Just one anecdotal datapoint.

</comment><comment><USER26> Switched my 4" smartphone and 8" tablet for a 6" phablet (Ascend Mate) a few years ago and it was the best thing I did.

Long battery and I can use it for most things I did on a tablet.

But with mobile VR coming along right now, everything seems to be optimized for 5" smartphones. I would love to buy a P8 Max, but depending on the Oculus Rift pricing, my next phone will probably be a <6" thing :\

</comment><comment><USER27> iPhone 6(S) Plus is way under-supported by the developers. It could display a split view just like the iPad, but check out how many apps take the advantage of this, see facebook, skype, youtube, etc. None of these. Twitter rotates landscape at least, but that's not support, just a feature ticked in Xcode.

</comment><comment><USER28> > This trend is driving other trends like the rise of consumption activities > on the phone:

I think the causation is backwards here. Growth in activities where a large screen is useful is driving the adoption of large-screen phones, surely?

(Feeling unreasonably pleased as this is one tech trend I managed to call and was ahead of the curve on)

</comment><comment><USER29> Manufacturers are riding the large screen wave in my opinion because of two reasons:

1) Battery life is likely consumer's #1 complaint about their phones. If you have a big screen, you can pack a big battery under it. 2) "Arms-race" product differentiation ala "Mhz wars" on computers. "My screen is bigger than the last model..."

I miss my old Palm Pre phones. I don't need a big screen for checking my email and Facebook, doing some chats, some voice calls, playing some music, and reading the odd news article, which is what I use my phone for.

But as a consumer I am being forced down the large screen path, because that is all the market is offering for some time now.

</comment><comment><USER30> I am somehow reminded of a snippet of a Jesse Schell talk back in 2010 [1] when he predicted convergence of technology in the pocket ... I, too, suspect devices are going to converge towards a device that can be held with both hands, has maximum screen real-estate and still fits in a pocket. One hand usage seems less important as the hold-to-ear phone aspect is slowly phased out as a valid use-case for these devices.

[1] [http://www.realtimetranscription.com/showcase/DICE2010/Jesse...](http://www.realtimetranscription.com/showcase/DICE2010/JesseSchell/index.php)

</comment><comment><USER31> Link is dead with a "Error establishing a database connection" message.

I'm not familiar with "avc.com", but I imagine that any entity with a three- letter domain name must have spent a fair amount of money establishing a web presence. So whichever CMS platform runs their site, how are they NOT using a standard cache plugin that pre-renders pages for articles?

There's no way that a professional CMS-driven website should completely go down over a broken database connection. My personal blog site only draws around 10K page views per MONTH, I only spend $5/mo hosting it, and this sort of outage wouldn't be possible.

<comment><USER32> avc.com is Fred Wilson's (a venture capitalist) personal blog. It gets regularly linked off of HN, so i'm surprised that it's down.

Judging by the cached version: [http://webcache.googleusercontent.com/search?q=cache:zpcZCGt...](http://webcache.googleusercontent.com/search?q=cache:zpcZCGts6iMJ:avc.com/2016/01/the- phablet-era/+&cd=1&hl=en&ct=clnk&gl=us) It's a simple static blog entry with disqus comments. There's no good reason for it to be hitting a database on every request.

</comment></comment><comment><USER33> "We are converging on a single device format in mobile and that's driving some important changes in usage. We are in the phablet era."

I don't believe the data Flurry presented can lead us to say that the device format is driving changes in usage.

I believe that looking at usage, per device, you're actually just segmenting the usage of the demographic that purchases that device.

For example, Fred has a graph that shows medium sized phones have a 100%+ increase in Sports usage. That may be due to the fact that most medium sized phones are being bought by middle aged men (who drive the majority of sports usage)

</comment><comment><USER34> There's a little problem with the article:

> The most interesting stat I saw in the report is this chart about device > distribution in 2015:

> It's very much a normal distribution centered around the 5.5″-6″ mobile > phone (phablet). There are still some people out there using smaller mobile > phones and small tablets

Actually the graph shows the _growth_ of time spent on devices, not total distribution of time spent. I'm sure total time shows quite a different thing than "some people" using small phones.

Of course the rest of the post is still correct, though.

</comment><comment><USER35> It would be great if these phablets could make a room for smaller phones that don't suck. We already see proper "mini", "compact" phones, so let's hope they actually get mini and compact. Right now they would be laughed off as too big in the early days of smartphones.

</comment><comment><USER36> Cached version: [https://webcache.googleusercontent.com/search?q=cache:http:/...](https://webcache.googleusercontent.com/search?q=cache:http://avc.com/2016/01/the- phablet-era/)

</comment><comment><USER37> We suspected this was going on when we looked at the analytics for our ecommerce sites recently and noticed a big shift away from tablets towards mobile (Google breaks them out separately). Large phones would of course fall under the latter category.

</comment><comment><USER38> I think this is largely due to "bigger is better" mentality of marketing (and consumers). All it will take is one OEM to make a popular smaller phone and all the other OEMs will start marketing to this "new segment".

<comment><USER17> But bigger really is better when it comes to displaying content. Sure at some point handilyness begins to suffer, but it seems there is a sweet spot at 5.5". I wouldn't want a bigger phone at that point, but I don't want to go back to a smaller one either.

</comment></comment><comment><USER39> The HN effect. MySQL is down at avc.com

</comment><comment><USER40> In less developed countries where power is unreliable, small simple phones with long battery life dominate. The market demands a phone that can go at least three days and ideally five before requiring recharge. They can't charge power hungry phablets every day. Half or more of humanity live in these places. That should be taken into account when looking at these statistics.

</comment><comment><USER41> This is all well and good, but in 3 to 5 years, VR/AR will make the screen go away completely. The shape of your device will be purely a fashion statement over any sort of feature requirement.

And yes, I think there will come a point where people will want to be seen with a headset on. It will be a form of conspicuous wealth signalling, like smartphones to begin with, smartwatches now, or long fingernails millennia ago. After the initial growing pains of "ugh, what an asshole", it will normalize and then everyone will do it.

<comment><USER42> Like how everyone wears bluetooth earpieces now, right? :P If I recall correctly, bluetooth earpieces went through a 15 minutes of fame "oh cool" public reaction followed by un-ending "ugh, what an asshole" as everyone had the experience of answering someone saying "hello" at the grocery store and then realizing they were talking on their bluetooth.

<comment><USER41> The problem is that Bluetooth headsets are universally shitty devices. You _only_ wear them to be conspicuous, and by now everyone knows the emperor has no clothes.

There is a lot more money going in to HMDs than Bluetooth earpieces. I would not be surprised to see an AR headset, about the size of large pair of aviator sunglasses, on a model on a fashion runway in 2 years.</comment></comment></comment>


Rich kid's successful defense in DUI deaths:'Affluenza' - <AUTHOR> http://jacksonville.com/breaking-news/2013-12-11/story/affluenza-defense-works-rich-teen-gets-probation-after-killing-4

<comment><USER2> This is a consequence of the significantly different treatment defendants receive if they are classified as "juveniles" (that is, not adults). Yes, drunken driving that results in the deaths of pedestrians is usually a criminal offense, all over the world. However, in many places, and for sure in every state of the United States, a minor who commits a criminal act will not be put on trial in the adult court system, but rather in a special juvenile court with different rules and different outcomes for a case. The usual outcome of a juvenile court case is court supervision to try to help the juvenile grow up to be an adult who won't commit crimes. That is the disposition that was ordered in this case.

Because of the severity of the charged offense, the prosecutors in this case asked for, and apparently were within the scope of juvenile court rules to ask for, hard prison time if the juvenile defendant was found delinquent. But, yes, in this case the defense lawyers, who were no doubt expensive specialist defense lawyers, came up with a defense that convinced the judge to order a less harsh disposition for the case. A news report I saw[1] included this report about what a defense lawyer said after the case decision:

"'There is nothing the judge could have done to lessen the suffering for any of those families,' said defense attorney Scott Brown, CNN affiliate KTVT reported.

'(The judge) fashioned a sentence that is going to keep Ethan under the thumb of the justice system for the next 10 years,' he said. 'And if Ethan doesn't do what he's supposed to do, if he has one misstep at all, then this judge, or an adult judge when he's transferred, can then incarcerate him.'"

So it could be that the boy will still do prison time (which I personally think would be more just than the treatment disposition he has received so far). Some of the facts in another report about the case[2] are really stark. This boy was a disaster waiting to happen in his earlier behavior, and his parents appeared not to do much about that. That's just wrong. But the prosecutors had their one chance at trial to ask for a tougher sentence on the boy, and the trial court decision now cannot be undone to allow them to ask for a second trial to get tougher punishment. Nor can the parents be tried for anything under usual principles of United States law in this case.

[1] [http://edition.cnn.com/2013/12/11/us/texas-teen-dwi- wreck/](http://edition.cnn.com/2013/12/11/us/texas-teen-dwi-wreck/)

[2] [http://www.wfaa.com/news/crime/Defense-pushes-for- intensive-...](http://www.wfaa.com/news/crime/Defense-pushes-for-intensive- therapy-for-teen-in-drunken-crash-that-killed-4-235288101.html)

<comment><AUTHOR> Thank you for the added context. Sadly, it appears that DUI is treated rather lightly in Florida, it's not just in cases of juveniles. For example: [1]Person driving with BAC greater than twice the legal limit kills a pedestrian. They got 1 year house arrest and then probation. [1][http://www.orlandosentinel.com/news/local/pedestrian- deaths-...](http://www.orlandosentinel.com/news/local/pedestrian-deaths- central-florida/os-pedestrian-vignette-stout-20130703,0,7342662.story)

</comment></comment><comment><USER3> His therapy is already off to a great start - the judge letting him suffer virtually no consequences for quadruple manslaughter will surely convince him that the belief that he can get away with anything is misguided.

<comment><USER4> To be taken with a grain of salt, but I read some local newspaper comments, including some from people who know the family/business state that not only that, but the level of remorse shown was approaching zero. Or to quote the commenter I read, "His IDGAF levels were pretty high. He still posts on Facebook about partying and drinking, it's just much more locked down privacy- wise".

</comment></comment><comment><USER5> It will be interesting to see the civil suits. The parents apparently argued that their son had no sense of personal accountability because they'd always given him anything he wanted and never told him no. When damages start being calculated for the victims and their families, there's a pretty good argument that it's not safe to leave the parents with any money.

<comment><USER6> My thoughts were similar: if the parenting is assigned most of the blame, maybe the parents should be serving the jail term? (I'd hate that as a general precedent – even good parents can yield a criminal child – but if in this case the parents have effectively embraced that culpability...)

Perhaps civil suits, leveraging the criminal case argumentation, can approximate that outcome.

<comment><USER5> I'd love that as a general precedent. Either the kid is tried as an adult, or the parents are tried in their place. It should be the parents' job to protect society from their children.

</comment></comment></comment><comment><AUTHOR> More Detail: [http://www.orlandosentinel.com/features/blogs/gone- viral/os-...](http://www.orlandosentinel.com/features/blogs/gone-viral/os- rich-teen-dui-probation-12112013,0,1975251.post)

</comment><comment><USER7> There's basically a separate system of justice in most countries for anyone with a net worth of 8 figures or more. He literally got away with murder.

<comment><USER8> This reminds me of:

      http://www.snopes.com/politics/bush/laura.asp

</comment></comment><comment><USER9> Is there any doubt the judge was paid off? No sane person could make an argument that a serial criminal, a __serial killer __, should not face consequences for his actions because he didn 't face any in the past.

<comment><USER10> >a serial killer

He's merely a parallel killer.

</comment></comment><comment><USER11> If he was black he would spent 2 years in juvi then shipped off to life in prison.

How can a community go on after this.

</comment><comment><USER12> Alright proletarians, back to work. Nothing to see here. And file your taxes on time this year!!</comment>


Reaching new records in speech recognition - <AUTHOR> https://www.ibm.com/blogs/watson/2017/03/reaching-new-records-in-speech-recognition/ <comment><USER2> I'm responsible for the Watson Speech JS SDK, which is aimed at making the speech services easy to use in web apps.

Code is at [https://github.com/watson-developer-cloud/speech- javascript-...](https://github.com/watson-developer-cloud/speech-javascript- sdk)

Simple demos at [http://watson-speech.mybluemix.net/](http://watson- speech.mybluemix.net/)

More complex demo at [https://speech-to-text- demo.mybluemix.net/](https://speech-to-text-demo.mybluemix.net/)

I'm goin to be out some this evening but feel free to ask me questions and I'll answer them as available.

<comment><USER3> What's up with the lack of https and encryption in IBM's cloud? I'm sure your machine learning stuff is great and all but the lack of proper encryption and security measurements makes it a total no-go. How can you expect anybody to take your cloud services seriously when you do http in 2017? I'm asking as someone who had their boss talked into trying out ML stuff one Bluemix by your sales people.

<comment><USER2> Can you give me an example?

[http://stream.watsonplatform.net/](http://stream.watsonplatform.net/) (the domain that the speech APIs use) redirects to https. Ditto for [http://gateway.watsonplatform.net/](http://gateway.watsonplatform.net/) which is what most of the other APIs use.

Both of the linked demos also redirect to https if you try a http URL.

<comment><USER3> Yes, most of your links (just like many of IBM's cloud related web services). If anything you, IBM and any potential customers, should be seriously concerned that they begin with [http://](http://) in the first place.

</comment></comment></comment></comment><comment><USER4> We have been a pretty large user of this feature within Watson for the last 6 months... while it is pretty good, it lacks the ability to take external inputs such as stereo recordings with channel markers. I've been working on migrating our solution to voicebase whom in my opinion has a much more robust solution when compared to ibm with respect to speaker diarizarion specifically because they include a feature to do channel markers. The result is a conversational transcription being much easier to read. Prior to this we used the Lium project to attempt diarization on a single channel recording. We had mixed results. Without a doubt in the last 12 months speech to text has rapidly improved

<comment><USER5> Why migrate to another service in 2017 when open source toolkits like Kaldi provide you both better results and more features and no vendor lock-in.

<comment><USER4> Cool - well we are hiring, so if you'd like to do this reach out. We have lots of neat projects like this going on all the time.

</comment></comment></comment><comment><USER6> I wish they clarified whether this claim that humans have a 5.1% error rate is in "listen to this sentence once and transcribe it" or "study this recording however you like and transcribe it."

edit: They talk about this in the arxiv paper:

>The transcription protocol that was agreed upon was to have three independent transcribers provide transcripts which were quality checked by a fourth senior transcriber. All four transcribers are native US English speakers and were selected based on the quality of their work on past transcription projects.

>...The transcription time was estimated at 12-14 times realtime (xRT) for the first pass for Transcribers 1-3 and an additional 1.7-2xRT for the second quality checking pass (by Transcriber 4). Both passes involved listening to the audio multiple times: around 3-4 times for the first pass and 1-2 times for the second.

<comment><USER7> For anyone wondering what the recordings in the HUB5 2000 eval data (the test data) sound like: [https://catalog.ldc.upenn.edu/desc/addenda/LDC2002S09.wav](https://catalog.ldc.upenn.edu/desc/addenda/LDC2002S09.wav)

<comment><USER8> God. Transcribing that would be mind-numbing. Glad computers are getting better at this.

</comment></comment></comment><comment><USER9> I spent a hack weekend recently using this API as part of a project to help with collaborating on usability testing reviews.

Upload a video, it strips out the audio, pushes to Watson for transcription, converts the result to a caption/subtitle track, and then allows people to comment on the discussion (like Google Docs).

I plan to polish it up a little and open source it soon.

<comment><USER2> Yes, please publish this!

Additionally, if you're using WebVVT format subtitles, I'd be interested in merging that code into the appropriate SDK. They're all on GitHub if you'd like to send us a PR: [https://github.com/watson-developer- cloud/](https://github.com/watson-developer-cloud/) (I'm on the team that is responsible for these.)

</comment></comment><comment><USER10> It would be great to see one of these frameworks offered as an offline solution. Right now the only options are WSR/Sphinx4, which have significant accuracy issues, and Nuance's products, which are extremely unfriendly to developers.

<comment><USER11> Mozilla's DeepSpeech has a native client [https://github.com/mozilla/DeepSpeech](https://github.com/mozilla/DeepSpeech)

</comment></comment><comment><USER12> It's great to see the improvements in this area. The voice recognition (if not the natural language processing) in systems like Amazon's Echo are pretty decent at this point for basic commands.

That said, I've tried the computer speech to text systems for transcribing interviews. And even with just one person talking they're nowhere good enough for me to use. Even budget human transcription (e.g. CastingWords) is just so much better that it's nowhere worth my time to use a machine-based system.

</comment><comment><USER13> Tl;dr: they stapled a 6-layer bi LSTM to Wavenet. Good to see IBM admit that deep learning is better than "Watson".

Also, with wavenet in the mix, no way this is used in production.

<comment><USER14> Also, these are cost prohibitive to use in apps widely or without limitations. Currently, Google Speech to Text is relatively cheaper at $1 for 166 messages. Source: [https://cloud.google.com/speech/pricing](https://cloud.google.com/speech/pricing)

<comment><USER15> Interesting. I worked for a company that did retail speech recognition in the late 90s, I should look up how much that cost in comparison to see how the economics are shaking out.

<comment><USER16> We'd be interested in that cost comparison!

<comment><USER15> Found something:

* $29.95 to register

* $9.95 per month subscription which entitles you to $14.00 worth of free transcription per month

* $3.50 per page (double-spaced, 225 words) for any pages in excess of your $14 allocation

$14 is four 225 word pages, so, 1-900 words for $0.11 per word, then $1.55 per word. Ish.

<comment><USER16> Ahh interesting. I like the subscription model from the perspective of the business. Seems like it would prohibit signing up for the low-volume user though, right?</comment></comment></comment></comment></comment></comment>


Ding.io - <AUTHOR> https://ding.io <comment><USER2> A product tour/screenshots would be helpful in evaluating what this product has to offer. Having that might increase your signup rates.

<comment><USER3> Totally agree. We're working on a new promo page at the moment, and it'll show off the actual product (w/screenshots) way better than this one.

<comment><USER4> Right now it looks like $10 or $25 a month for a stop watch app. If I was a customer this one would have scared me away. I'm on my phone if that makes a difference.

<comment><USER5> They have a 45 day free trial[1]

Suggestion: add text below the green signup button saying, "Free 45 day trial, no credit card required". I think that would also help increase signup rates.

[1] [https://news.ycombinator.com/item?id=7246383](https://news.ycombinator.com/item?id=7246383)

<comment><USER6> I'd suggest, if there isn't enough signups happening, free for solo usage, cost for teams might be the way to go. Get people hooked using it for their own time to the point they make work pay for it for the whole team.

</comment></comment></comment></comment></comment><comment><USER7> The homepage has no JS other than analytics and cloudflare, and css/html is super clean. Quite a nice departure from the typical bootstrap+jquery combo.

<comment><USER3> I love the fact that you inspected the code :)

We're fans of Bootstrap and jQuery ourselves, but believe in giving Ding the custom treatment. Time tracking is a crowded space after all.

</comment></comment><comment><USER8> Just downloaded, it looks great. Great design, really pleasing to use and look at. Lots of small attention to detail that makes it feel nice (eg, calendar picker with large trailing dates into future and past selected month).

One note: I'm interested in just using for time tracking for personal productivity, but it feels more like it's built around use cases for consultants / free lancers (eg, client name being required when creating project). Not sure if my use case is your target, but if it is might be nice to have a global setting to turn off the 'client / freelancing' features so i get something a little more streamlined and limited to better capture how I want to use it. Might diverge from what you're going for, but wanted to pass along how I planned on using.

<comment><USER9> +1 for this. There is not much out there for personal productivity time tracking, especially with good design/UX.

<comment><USER10> +1 on this one as well. I built a shell-based one that pipes out to Google Calendar [0] and then started using Toggl [1]. Toggl works best since it's mobile and web, and they sync with each other, but it's still not totally great from a UI/UX perspective. Timely [2] is beautiful, but doesn't have actual time-of-day data, which I'd like.

Nowadays, I simply plan out my day in my calendar, adjust as necessary, and take that as the data. I don't get to do any crunching of the data like I used to (like graphs and quantitative analysis of where my time is going), but it does serve the purpose of making me aware of what exactly I did during the day.

I'm working on something that will do this quantified-self-esque time tracking built specifically for QS (instead of for freelancers) when I have free time, because this awareness of how I spend my days has quite literally changed my life.

[0] [https://github.com/markbao/vestige](https://github.com/markbao/vestige) [1] [http://toggl.com](http://toggl.com) [2] [http://timelyapp.com](http://timelyapp.com)

<comment><USER9> Totally agree re: Toggl (UX issues really bugged me -- time tracking can and should be low friction) and Timely (pretty but flawed interface).

Also totally agree re: quantified-self time tracking and its usefulness.

</comment></comment></comment></comment><comment><USER11> The UI is nice, but I think nothing beats automation. I use RescueTime [1], which is surprisingly _accurate_ measuring my time. I use it for years now and I'm very satisfied with it. No need for "administrative overhead".

[1] [https://www.rescuetime.com/](https://www.rescuetime.com/)

<comment><USER12> Remember when someone posted their free-for-use stock photos of Apple products and a lot of people talked a lot of smack about his photos? It's interesting how I see them on a lot of product pages now.

<comment><USER13> I love Unsplash, who's talking shit?

<comment><USER12> [https://news.ycombinator.com/item?id=5794083](https://news.ycombinator.com/item?id=5794083)

:(

</comment></comment></comment></comment><comment><USER3> Noticed a lot of activity from this link just now – exciting to see you guys checking out our time tracking tool. We really appreciate feedback from the community, so don't hesitate to comment here if you have feature requests or thoughts on Ding.

Gjermund, Co-founder / Designer

<comment><USER9> Three quick thoughts on some details:

1) Please, PLEASE make sure it's quick/easy to enter time with the keyboard. Right now the main log time interface doesn't let you tab to select the project drop down. I feel like I should be able to tab to this field after the optional comment field. Enter anywhere in the form (except maybe the optional comment) should save the entry and get me ready to enter a new one. For your target audience (devs), I think good keyboard-only support is key.

2) There should be an easy way to see the total time for each day, maybe just next to the date on the Activity page. In any case, I shouldn't have to run a query to see how much I've worked today.

3) Have queries (via the Overview tab) alter the URL so a query can be bookmarked. Ideally this would be with relative dates so I could bookmark [https://ding.io/overview?user=1&project=x&client=y&time=this...](https://ding.io/overview?user=1&project=x&client=y&time=this_week&billed=false).

This looks really good. Harvest is the de facto for time tracking with good UX, but even their UX could use some refinement (my 3 comments above all apply to Harvest also, I think). You're definitely giving them a run for their money.

BTW, I really like having the client and project in one searchable drop down. Separating these was one of my biggest annoyances with Harvest.

<comment><USER3> That's some great tips. Thanks!

1) Agreed. This has been up for discussion several times, and it's high on the list. We'll post these kind of smaller tweaks on Twitter, so if you stick to using Ding, keep an eye out there [0].

2) We've tried to get the most essential stats within the statistics panel on the top, and let you quickly tab through these. The hours of the day is usually pretty visible within the top of the activity feed, but I guess it is less clear if there's several short intervals rather than two-three big ones. I'll take a close look at it and see if there's a way to tweak this.

3) Another great idea – we'll see if we can move it further up the list.

We really appreciate all the great input, so thanks again. Our biggest challenge is time as we're a small team, but we'll keep on truckin' :)

[0] [https://twitter.com/ding_io](https://twitter.com/ding_io)

</comment></comment></comment><comment><USER14> Would be useful if you included screenshots without having to sign up.

<comment><USER3> Agreed. We're working on improving the promo page at the moment, and will make sure we include several screenshots.

</comment></comment><comment><USER15> The product itself looks really nice, but the pricing model doesn't really seem like a good fit. I think a freemium model would work better than a subscription based one.

<comment> shawn-butler Or include invoicing for the price. By itself time tracking isn't worth $10 month to me.

From the work so far, I look forward to what they could do to simplify invoicing.

<comment><USER3> I take that as a compliment, thanks. We have no plans of changing the price, so every new feature we add will be included into the existing plan, giving you more bang for the buck.

There's been quite a few requests regarding invoicing, and while I won't promise anything in the short term, we'll work hard to make Ding better in the future. People requested a start/stop-timer which we just implemented, so we do listen and take your feedback seriously.

</comment></comment></comment><comment><USER16> It looks nice. I'm a user of freckle for some time so I recognize quite a lot of the UI and features. Even if I don't use all features of freckle, some are quite useful. As an example in freckle I use tags and the API to automatically export all entries every month/week updates the client through email reports or google sheet updates. It doesn't look like that would be possible with ding.io yet (exkluding the manual csv download)? Also the dashboard display priority seems a little off.

Money earned gets most space however is never the truth because of taxes, expenses etc. I rather use my accounting software for seeing my actual earnings.

Time logged is OK however Active projects for me isn't really necessary information.

The activity stream and log time form should be of highest priority. Put the activity stream in focus with better one-click-filtering on clients/project/tags(?) would be great to quickly answer the clients questions on what I did. :-)

In summary I think the design is great and looking at features I think when comparing with freckle, which is relevant in my case, I feel like half the price equals half the functions. Nothing wrong with that and your product will of course mature with time. I'll keep an eye on it.

Great job!

<comment><USER17> Thanks for the kind words and thorough feedback!

You can export to both CSV and PDF. So you can still get reports for your clients, although you'll have to do it manually. We have a pretty limited API right now, built mostly to support our iOS app. As the API grows you should be able to get the reports from it.

Money earned let's you see how much you can bill in a given period, so you can quickly see if you're reaching your goal. We think it's useful despite not accounting for taxes and expenses, and it's a nice moral boost to see the money coming in.

You can filter on projects, clients, date etc. on the Overview page. It's really fast to get filter out exactly the data you need.

</comment></comment><comment><USER18> I use Harvest now, their OSX app actually tracks when I'm idle for too long and will prompt me to roll back the amount of time I've been idle. It doesn't remind me to turn on the timer though when I forget. ;-)

</comment><comment><USER19> I've been a happy subscriber of Toggl for some time now (2+ years). I find Ding's pricing to be higher, and I don't see any integration with other services (like FreshBooks, etc..). I see I can export data, but does anyone know if they plan to integrate with other services?

<comment><USER3> If we get numerous requests for a certain integration, we'll see what we can do. Until now, there haven't really been much of this, so we've put our efforts into the developing the product itself.

As for the price, I guess it's higher than some, and cheaper than others. We've tried to set a price we thought was fair, and we continuously keep making the product better to make sure you'll get more bang for your buck.

</comment></comment><comment><USER20> Congrats on the launch! This looks great! I started with something similar but targeted more toward time tracking in organizations - [https://github.com/prakhar1989/timelogger](https://github.com/prakhar1989/timelogger)

</comment><comment><USER21> Good job! I like the interfaces. Haven't found option for piece-rate projects though. Do you have any plans on the matter ? Another guess: I think some API features would be extremely useful for such a service.

<comment><USER17> Thanks for the feedback, I agree it would be useful to have the option to set a fixed price for a project. We actually did some wireframes for this a while back, but we never got around to implementing it. Maybe now would be a good time to look at it again.

What API features are you thinking about? We have a pretty limited API that we built for the iPhone app, but it's growing as we work on the next version of the app. So over time it'll get more of the same features as the website.

<comment><USER21> Thank you for the answer! I see two directions for API: one is an integration with existing services api such as RescueTime, and the other is own api to give such opportunity to fellow developers. For example any task manager service can use time-tracking and project/profit management functions at your service via API.

</comment></comment></comment><comment><USER22> I agree with the other people in this thread, this is a beautiful looking app. I think one thing you may want to clarify is whether this is available in a browser, as it seems that this is an app to download.

</comment><comment><USER23> I've just started using Harvest but I like what I see here, too. I'm not sure there is enough of a difference (price, features) to make me switch but I see potential and the price is competitive.

<comment><USER3> You're more than welcome to give Ding a shot. It's free for the 45 first days, so you have nothing to lose :) I think you'll appreciate its clean UI and simple approach, and if there's anything keeping you from using it, shoot me an email at firstname.lastname@example.com and I'll see what we can do.

</comment></comment><comment><USER24> This looks really fantastic. I love the design and after spending a few minutes reading your blog it seems like you care a great deal about every decision you make.

<comment><USER17> Thank you! It's hard to compete on features alone, so we try to sweat the details.

</comment></comment><comment><USER25> I like the happy hippo behind the mba on the landing page :)

All in all its a great introduction to the service, really well put together.

<comment><USER3> I'm glad to hear you like it. Thanks! The hippo is a Danish design classic from 1955 made by Kay Bojesen: [http://www.kaybojesen- denmark.com/](http://www.kaybojesen-denmark.com/)

</comment></comment><comment><USER26> Just signed up. Looks nice. But kind of creepy that I had a profile pic on the site without having to upload one.

<comment><USER27> Gravatar maybe?

<comment><USER26> Yes. Thanks. Completely forgot about that.

</comment></comment></comment><comment><USER28> Well done. Any plans for an Android app?

<comment><USER3> Thanks! Making native apps is a pretty costly thing for a small team like us, and to be honest, we don't have much experience with Android development. So I can't promise anything at the moment, but we'll make sure to keep improving the responsive web. It's definitely on the wish list though.

<comment><USER29> Props for not promising that it's "coming soon" \-- vaporware sucks.

<comment><USER3> Haha, we actually have a rule of never using the 'coming soon' phrase :) We've done it in the past, and it always looks bad if you don't deliver on the promise.

</comment></comment></comment></comment><comment><USER30> Really nice! Very simply and clean. Integration with an invoicing system would be a great win.

<comment><USER3> So nice to hear, thank you! What kind of invoice integration would you want?

</comment></comment><comment><USER31> Love the idea. It is dead simple and works well. Going to show to a few freelancer friends.

</comment><comment><USER32> kudos, the site has a clean interface. Though I have no idea how to use the site properly. Is there any tour guide to the side or a simple welcome page on how to use it?

<comment><USER3> I'm glad you like it. Thanks!

I agree that a "get started" page could be a neat thing, and the onboarding experience is something we're planning to overhaul. Ding should however be quite simple: create a project and log time. As you log your hours, you can easily see the time spent and the money made.

If you want to filter out and export your time entries, it's possible in the Overview section.

I hope you'll give it a shot. If you're stuck, shoot me an email at firstname.lastname@example.com and I'll get to you asap.

</comment></comment><comment><USER33> Nice idea. Very clean looking website too.

</comment><comment><USER34> Grr, no pinchzoom across their website

</comment><comment><USER35> looks very promising!</comment>


An Anti-College Backlash? - <AUTHOR> http://www.theatlantic.com/national/archive/2011/03/an-anti-college-backlash/73214/

<comment><USER2> Posting this because it reminded me of a tweet by Don Dodge earlier where he said "36,000 high schools in USA, or 360,000 top 10 students. They can't all go to Harvard or Stanford. Only 2,600 4 year colleges. Do the math" (<http://twitter.com/DonDodge/status/54617148722655232>). Compound this with VCs offering to fund students to postpone their education and I do think it is fair to legitimately question (or challenge) our traditional and even conventional education system. Would love to hear thoughts from the HN community and how this applies to startups/tech/entrepreneurship.</comment>


Ask HN: Does native compilation still matter? - <AUTHOR> Have we reached the point where the overhead of a VM is acceptable for 99% of applications? Will native code become the domain of a few experts who write higher level environments used by the majority of programmers? <comment><USER2> Of course it still matters. Almost all the software that runs whichever device you're reading this post on has been compiled to machine code. In operating systems everything from the filesystem to device drivers to the window manager to the web browser are written in some low level language. For hard real-time systems you need to be close to the metal. Same for most embedded applications.

Sure, there are many situations where sacrificing performance in exchange for a VM is totally appropriate. If you think this covers 99% percent of applications (or even 50%!) then you may only be thinking of a very small subset of software development.

Also, inefficient code produces excess heat. Meaning reduced battery life and noisy fans. So all software that gets used a lot should be optimized for performance just to conserve energy.

<comment><USER3> > If you think this covers 99% percent of applications (or even 50%!) then you > may only be thinking of a very small subset of software development.

This statement was true in the '90s. Nowadays, the amount of high-level software written as a ratio of total programmer-hours is probably upwards of 90%.

Out of the hundreds of thousands of programmers out there, very few of them write kernels or drivers. Even fewer write window managers—there are maybe a dozen of them in active use, and most of them have either zero or one maintainers. Four software companies, out of thousands and thousands, write Web browser engines; two of them share most of their engines' code. With few exceptions, all of the codebases I mentioned date back to the '80s or '90s, when most of the code was written.

People seem to think that the software stack on a typical desktop OS is constantly under heavy development. It's not. Most of it is in maintenance mode or effectively unmaintained.

Don't take my word for it—look at the "who is hiring" threads on HN lately. There's some C++ in there, sure, but nowhere near the figure you quoted.

<comment><USER4> Totally ignores scientific applications, which there aren't enough supercomputing hours to go around for. NERSC's Cori, one of the top 3 supercomputers in the world, often takes a day to start a job.

Not to mention private companies developing software to analyze heaps of data from finance to web searches.

<comment><USER5> What's your source for it and why would it take so long?

</comment></comment></comment></comment><comment><USER6> I don't generally want to give up performance. As a game developer, I usually actually need all the performance I can get. My default choice is always a language with nice, low-cost abstractions, that compiles to native code. I don't see that changing, probably ever. I always want to get the most out of the machine.

<comment><USER7> What do you think of Rust?

</comment></comment><comment><USER8> Interesting question, but ultimately it's still hard to say .

The story on power consumption of managed/VM language is still not clear. Most of the computation in the world are now happening in mobile/IoT devices and cloud/data centers; Both environment put a strong constraint on power consumption and therefore on efficiency. There is already a huge amount (and growing ) of native code running in the cloud. As our computational requirement increase (and unless we have a break through in computation efficiency) people might want to revisit the lost implied by managed environment.

The other side of the coin is that managed env. are popular because they usually bring safety, ease of programming and productivity. Modern C++,rust and D are showing that a lot of those attributes can be had with native languages too.

</comment><comment><USER9> Yes we have. The majority of all commercial programmers work with web sites or web applications in which VM:s reign supreme.

But for hobbyist work, it is different. Free software developers are seeking universality more than anything else. They want as many people as possible to be able to use their code, be able to read it and contribute to it.

VM:s hinder universality. For example, if you write an image processing library in C then everyone can theoretically use it (assuming bindings are non-trivial to write). But if the same library is written in a VM language, then it is only usable to developers who are using that platform.

It's like a curse.

</comment><comment><USER10> As long as the man hours invested cost less then a more potent cpu in all devices sold

</comment><comment><USER11> With many dynamic languages having ahead-of-time and just-in-time compilation it's a murky question now.</comment>


The Story of a Great Monopoly (1881) - <AUTHOR> https://www.theatlantic.com/magazine/archive/1881/03/the-story-of-a-great-monopoly/306019/ <comment><USER2> Far too much here for easy synopsis. Picking two arbitrary items:

 _The contract is in print by which the Pennsylvania Railroad agreed with the Standard, under the name of the South Improvement Company, to double the freights on oil to everybody, but to repay the Standard one dollar for every barrel of oil it shipped, and one dollar for every barrel any of its competitors shipped._

Strong shades of Microsoft's per-CPU licensing agreement for PCs.

Or of how to respond to questions under inquiry:

 _When Mr. Vanderbilt was questioned by Mr. Simon Sterne, of the New York committee, about these and other things, his answers were, "I don't know," "I forget," "I don't remember," to 116 questions out of 249 by actual count._

The names change but the game's the same.

<comment><USER3> > The names change but the game's the same.

It's certainly true. Even though Gates was universally mocked for his approach, and although it was widely interpreted as making Microsoft look more guilty, it's exactly how Zuckerberg and Larry Page will respond when their embryonic anti-trust parties get to that point. Their lawyers will advise that that approach is still the safest way to go, despite how it will look. When in doubt, play dumb.

<comment><USER2> I actually had both general recent testimony and Reagan-era inquiries. The pattern is rather older, it seems.</comment></comment></comment>


How to invoice for design work - a quick guide - <AUTHOR> http://www.whatwasithinking.co.uk/2008/09/29/how-to-invoice-for-design-work-a-quick-guide/

<comment><USER2> Paypal as a payment service for a designshop?

That seems pretty absurd to me. A normal bank transfer is almost free and can be done by my grandmother to any account.

<comment><USER3> I'm curious as to what the alternative to BACS is in the US (and in other countries too, but the States particularly). Are there free and simple bank transfers there?

<comment><USER2> In Denmark you simply go to the bank, or log onto your netbank, and transfer money into another account. All you need is the account number. Couldn't be simpler. Isn't that standard practice in the US?

<comment><USER4> It is the same in the US. It is called an electronic transfer. You just need the account number and routing code for the bank.

</comment></comment></comment></comment><comment><USER5> This site doesn't seem to work on my iPhone.</comment>


Blood, Sweat, and Dialogue Trees: How Games Writing Has Evolved - <AUTHOR> http://www.glixel.com/news/blood-sweat-and-dialogue-trees-how-games-writing-evolved-w452099 <comment><USER2> If you're looking for a game with excellent writing, check out Pillars of Eternity. It's a modern Baldur's Gate/NWN with writing which won't make you roll your eyes.

<comment><USER3> I'll second this. Fantastic game, and so's Tyranny, which is by the same developers with a similar amount of great writing.

</comment></comment><comment><USER4> >"If there's anything I've learned, it's that game-making is a journey without a map," he says.

Very true. Or you're given a map, but it gets scribbled all over and/or thrown away and replaced on a weekly basis.

</comment><comment><USER5> They mention Gone Home in this, and I would highly recommend it. The writing is the best I've seen in a game, especially if you're into punk 90's riot grrrl stuff.</comment>


Finding secrets by decompiling Python bytecode in public repositories - <AUTHOR> https://blog.jse.li/posts/pyc/ <comment><USER2> Public Service Announcement.

While for the other secrets, I've got nothing, there is never a reason to have AWS secret keys in your code or in application specific configuration files.

Every AWS SDK will automatically read your keys from your config file in your home directory locally. Just run

      aws configure

When you run your code on EC2, Lambda or ECS, the same SDK's will automatically get the keys associated with the attached role.

<comment><USER3> For others non AWS credentials, just use environment variables

<comment><USER4> They have to be set up somewhere, though.

<comment><USER5> It should be done out of band.

Repo should have a `README` that says the secrets are in the team 1password account, talk to <team member> to get a 1password account, and get added to the group that has access to the vault with the creds.

The repo should have a source script that will pull the credentials[1] and `export` them to your ENV. `direnv` can make that happen automatically[2], or you can run that script from your `.bashrc` or similar.

You can do something similar with your favorite secrets manager. I've used a similar approach before, with good results.

[1] [https://support.1password.com/command-line-getting- started/](https://support.1password.com/command-line-getting-started/)

[2] [https://direnv.net/](https://direnv.net/)

<comment><USER6> If you use AWS, _"your favorite secrets manager"_ should be AWS SecretsManager, and you should use an AWS Role to limit access to the secrets.

Have your code fetch any secrets it needs from AWS Secrets manager, using the name of the secret (which does not have to be kept secret, so it can be in your source repo)

That way, you don't have to put secrets in your environment, with the risk of leaking them.

<comment><USER7> For comparison, open source password managers are zero cost, 1Password is fixed cost (even more fixed if one buys the software, instead of the subscription), and in contrast [https://aws.amazon.com/secrets- manager/pricing/](https://aws.amazon.com/secrets-manager/pricing/) is $0.40 per secret per month, plus a tiny but not zero cost per API access.

I'm just pointing out that AWS Secrets Manager is not at automatic, no-brainer win

<comment><USER2> SSM Parameter Store (not Secrets Manager) is also "zero cost" and you can have a parameter of type "secret string".

The other solutions don't integrate with AWS IAM. _Something_ has to grant access to the password vault. In the case of Secrets Manager/Parameter Store you just grant access to the role attached to your EC2 instance/ECS cluster/Lambda.

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> This would be solved if python used an (OS-specific) cache directory for its .pyc files. I have always disliked .pyc files... here's a concrete reason!

Question: what does python do if it doesn't have write permission in the current working directory? Not write the cache?

<comment><USER9> It already (since 3.2, as the article points out) uses a cache directory. Why would an OS-specific one help?

You can also set PYTHONPYCACHEPREFIX if you want to use 'a mirror directory tree at this path, instead of in __pycache__ directories within the source tree' \- [https://docs.python.org/3/using/cmdline.html#envvar- PYTHONPY...](https://docs.python.org/3/using/cmdline.html#envvar- PYTHONPYCACHEPREFIX)

The PEP mentions your case at [https://www.python.org/dev/peps/pep-3147/#case-5-read- only-f...](https://www.python.org/dev/peps/pep-3147/#case-5-read-only-file- systems) . But honestly, I don't really follow the answer. I think it means "ignore creation and write failures."

<comment><USER10> > _Why would an OS-specific one help?_

E.g., if you want to back up your home dir, and omit caches, since they can be regenerated. It's a lot easier if programs write their cache data to ~/.cache / $XDG_CACHE_HOME than if they intermix it / scatter it about.

<comment><USER9> But how does it help to have one directory for all Linux-based OSes (or one directory for RHEL, one for Ubuntu, one for Debian, etc.), and one for FreeBSD and one for OpenIndiana?

At least, that's what I interpret "OS-specific" to mean.

<comment><USER10> More along the line of the original comment you're replying to, if the cache was in, say, ~/.cache, then it won't get swept up in the repository's commits, since the cache data is no longer inside the repository's working directory. Then, it never gets uploaded to GitHub, and this security issue never happens.

I have seen a surprising number of people — some who are engineers by profession too, and ought to know better — just git add everything, and then commit it all without looking. One _should_ review the diff one has staged to see if it is correct, but alas…

<comment><USER9> That's possible with 3.8's PYTHONPYCACHEPREFIX, yes?

Perhaps it's worthwhile for someone to blog about this more/promote this as a best practice? Though what's missing is the hook to connected it as appropriate for the given platform.

I see now that "OS-specific" was meant to be interpreted as "the OS-defined mechanism to find a cache directory", not "a cache directory which differs for each operating system".

I would not have been confused by the term "platform dependent", which is what Python's tmpdir documentation uses, as in: "The default directory is chosen from a platform-dependent list" at [https://docs.python.org/3/library/tempfile.html?highlight=tm...](https://docs.python.org/3/library/tempfile.html?highlight=tmpdir#tempfile.mkstemp) .

</comment></comment></comment></comment></comment></comment><comment><USER11> Highly recommend "export PYTHONDONTWRITEBYTECODE=1" in your bashrc and just forget about it. Pyc files are still an important optimization on modern machines in some circumstances (especially with huge oft-restarted apps), but the autogeneration behaviour has always been a pain in the ass

The bulk of your pycs are generated during package install. What tends to remain in the usual case is a handful of files representing app code or similar.

<comment><USER12> Why do you highly recommend this? You can set git (or other VCS) settings to avoid committing .pyc files. They're not large files; I don't see the downside.

<comment><USER11> Difficulty with git basics is the least of your worries. For example pycs are fundamentally racy, it is quite possible to have a .py newer than a .pyc depending on how unlucky you were with an in-progress deployment, or some tool that never updated the .py timestamp. Python continues to execute the pyc even though the code changed, since the minimal benefit of the pyc would be rendered significantly inert should Python use any kind of strong check (instead of just comparing second-granularity stat() output) to ensure the cached bytecode matches the source. In this way, without your permission, Python silently plays undesirable code execution roulette with your computer every time it starts, for as long as you have the feature enabled.

I have lost count of the number of times I've seen someone lose an hour due to it. I can also count many instances of QA environments becoming inexplicably bricked by it. The correct fix for this requires opening the .py and hashing its content, at least doubling the amount of IO required to start a program. They were a great feature when parsing small files was noticeably slow, but this hasn't been true for almost 20 years.

It's therefore worth turning the question around: why do you think pyc files are useful?

<comment><USER13> In the last 12 years of writing python, I have only hit had issues with .pyc files a handful of times, and always with python < 2.7. Anaecdotally this experience is shared with everyone I have worked with.

If you're seeing this regularly, it suggests there may be something unique or uncommon in your set-up. You may wish to isolate and change whatever that is.

<comment><USER14> Now that you mentioned it, I just realized I never have problems related to .pyc files anymore ever since I switched to python 3 a few years ago. I remember I used to have problem with deleting database migration files because python would load the .pyc files of deleted migration scripts unless I also delete the .pyc files (which I often forgot).

</comment></comment></comment></comment></comment><comment><USER15> Just reading the script from TFA, it attempts to find secrets.pyc and decompile it, but doesn't even check if secrets.py is also in the repo. A glance at search results (I just used GitHub's web interface, didn't bother to run the code) tells me when secrets.pyc is committed, secrets.py comes with it at least the vast majority of time.

I guess the author did find cases where secrets.pyc is committed but secrets.py is not? It's hard to fathom how that could have happened (especially inside "organization" settings). Sounds like the result of absolute rookies in both Python and git following a tutorial with a step "add secrets.py to .gitignore" but unfortunately takes ignoring __pycache__ and ﹡.pyc for granted, which is too much to ask for some people.

> it is very easy for an experienced programmer to accidentally commit their > secrets

No, it doesn't take an experienced programmer to put __pycache__ and ﹡.pyc to global ignore, or use a gitignore boilerplate at project creation, or notice random unwanted files during code review.

<comment><USER16> I think I am an experienced developer (not Python) and this would never cross my mind.

<comment><USER17> It would never cross your mind not to commit .pyc files to source control? They're not even source. Committing .pyc files is to Python what committing .o files is to C.

<comment><USER15> > They're not even source.

To be clear, they're not even text. You don't need to know Python at all to realize something's not right when you're committing unknown binaries to source control.

</comment></comment></comment></comment><comment><USER18> Radare2 also supports Python bytecode of diffent versions [1].

[1] [https://github.com/radareorg/radare2/tree/master/libr/asm/ar...](https://github.com/radareorg/radare2/tree/master/libr/asm/arch/pyc)

</comment><comment><USER14> Another possible place to look for secrets is in public docker images. Bots are scanning github repos for secrets all the time, but what about dockerhub (and other docker images repositories)? I accidentally leaved a secret on my public docker image once and that's made me quite paranoid about it now.

<comment><USER19> I scanned everything pushed to dockerhub for a few weeks but didn't find too much interesting stuff showing up

</comment></comment><comment><USER20> I read "Finding secrets by decompiling Python bytecode in public restrooms" by accident. It never occured to me that anyone would do THAT in there.

<comment><USER21> there's bytecode sharpied on the walls</comment></comment>