Chirpss – Hear and see people visiting or leaving your website - <AUTHOR> http://chirpss.com/ <comment><USER2> "Chirp chirp" is the common onomatopoeia for cricket sounds, i.e. nobody is there. So it's funny to me that "chirpss" is used as the name of a product that announces when someone visits your website. You have literal chirps when someone is there, and figurative chirps when no one is.

</comment><comment><USER3> This is known as "Calm Technology", as first defined in " _Designing Calm Technology_ " by Mark Weiser and John Seely Brown in 1995: [http://www.ubiq.com/hypertext/weiser/calmtech/calmtech.htm](http://www.ubiq.com/hypertext/weiser/calmtech/calmtech.htm)

<comment><USER4> _"Placed in an unused corner of a hallway, the long string is visible and audible from many offices without being obtrusive."_

Ahhhhh must have been nice to live in 1995! :-)

</comment></comment><comment><USER5> Since I host my own website this is really easy. Here's how I did it in 2006,

tailbeep -f /var/log/thttpd.log -t /dev/tty1 -F 3000 -M 100 -s GET

But it's obvious how to adapt this to my modern nginx setup. You can get tailbeep at [https://soomka.com/tailbeep](https://soomka.com/tailbeep)

</comment><comment><USER6> I hear some guys at Google added this for Google.com for a joke. They're deaf, and in Arkham asylum now.

</comment><comment><USER7> It's the perfect excuse to get you to give up your analytics data to a third party

<comment><USER8> It's a bookmarklet, so you can see the code. It doesn't send any data anywhere.

<comment><USER9> Apparently you can only use it with Google Analytics, so it is indeed encouraging you to give up your analytics data to a third party. :)

<comment><USER10> Assuming you're not already doing that.

<comment><USER11> I'm not and I won't.

</comment></comment></comment></comment></comment><comment><USER12> Here is the code of the bookmarklet [https://gist.github.com/dustingetz/000c3d048bc5ed0c3f7650ac6...](https://gist.github.com/dustingetz/000c3d048bc5ed0c3f7650ac698125c0)

</comment><comment><USER13> Similar: [https://askmike.org/articles/ring-a-bell-when-someone- visits...](https://askmike.org/articles/ring-a-bell-when-someone-visits-your- website/)

There's also a Chrome extension [1] and a website [2] for the same purpose.

[1]: [https://chrome.google.com/webstore/detail/startup-bell- doorb...](https://chrome.google.com/webstore/detail/startup-bell-doorbell- for/pbjbgplnnelkehhpnlcfolhgdimmlehd)

[2]: [http://bellbot.com/](http://bellbot.com/)

</comment><comment><USER14> If you use a product like SalesIQ it creates noise in the dashboard as users enter and leave (and hit interim targets you define)

</comment><comment><USER15> Lol maybe for your personal website otherwise its going to be the most annoying thing in the world.

<comment><USER16> Every company got its start at some point.

</comment></comment><comment><USER17> I like the fact that it's temporary and doesn't require any modifications on the website, nor does it need any kind of server code, just a js bookmarklet you execute on your analytics page.

</comment><comment><USER18> TL;DR; script for Google Analytics, that chirps when people enter/leave your site.

</comment><comment><USER19> This isn't healthy.

</comment><comment><USER20> Not sure why this is #1 on HN, or why it was developed?

<comment><USER21> Because this is largely a webdev community, and who doesn't love instant notifications that someone's paid attention to you?

<comment><USER22> At the very least people shouldn't upvote posts with non-descriptive titles.

</comment></comment></comment><comment><USER23> Why? We already have too many alerts interrupting our lives. The best thing you can do is have fewer alerts, not more.

<comment><USER16> It's fun. Especially when you are first starting out.

</comment></comment><comment><USER24> The URL has ref=producthunt.

<comment><USER25> What does that mean?

<comment><USER26> It means that this link is misinforming chirpss that we were referred ("refered") by producthunt.

<comment><USER27> Is that not accurate assuming the OP is from or was referred by producthunt?

<comment><USER26> The OP was, but I was referred by news.ycombinator.com.

<comment><USER28> Sure, but they are for measuring the effectiveness of marketing channels, not simply the direct referrer. They can discover the direct referrer through HTTP headers. If a marketer promotes something on Product Hunt, then somebody on Product Hunt submits it here, then you follow the link from here, it means the marketing channel that reached you was Product Hunt. It gives marketers better information in aggregate about which marketing channels are most effective.

<comment><USER26> I'm glad to understand that, though it doesn't make me any more inclined to add any extra metadata to my web traffic.</comment></comment></comment></comment></comment></comment></comment>


San Francisco to Uber, Lyft: Tell us what drivers earn - <AUTHOR> https://www.sfchronicle.com/business/article/San-Francisco-to-Uber-Lyft-Tell-us-what-drivers-12951396.php <comment><USER2> Why now? Why wasn't this done several years ago?

<comment><USER3> Dennis Herrera is just trying to make a name for himself for when his re- election comes up on November 5th, 2019. Hating on tech is a re-election strategy.</comment></comment>


An Illustrated Guide to Guy Debord's 'The Society of the Spectacle' - <AUTHOR> http://hyperallergic.com/313435/an-illustrated-guide-to-guy-debords-the-society-of-the-spectacle/ <comment><USER2> I've felt endless frustration with Debord's and SI's ideas since I stumbled on them about 10 years ago. I know there is something hidden beneath this life, and these works are simply pointing in its general direction. For a long time I thought truth was encompassed by the concept of "authenticity", in terms of expression and being as a human. That idea of mine was struck down pretty swiftly by a simpatico social theorist I admired on Twitter, he responded to my question, "What is authenticity, exactly?" with the answer "Don't worry, nothing is real. Everything is constructed."

<comment><USER3> Should I live in the woods as a hermit? Not engage with consumerism? What do I do with this information?

<comment><USER2> These questions have been the main source of my frustration. The closest lifestyle I've encountered that jibes with these ideas is punk. Even then, taking on the identity of "punk" is like putting on a funny mask over your original mask and yelling, "Look everyone, it's really me!"

<comment><USER4> To speak with Debord: Punk has been subsumed within spectacular society.

Interestingly, Debord seemed to have proposed a life style of Derive, aimless wandering, following one's desires. To me, this resembles the mind set that he tries to attack. Both tell individuals "You are a snowflake! Follow your inner desires!"

Maybe the more during counter life style against the fads and suppressions of the spectacle society, is a devotion to objectivity and naked facts.

</comment></comment></comment></comment><comment><USER5> The problem with the Situationists is that they have _no_ political program. They refuse to describe how they would like the world to be, instead coming up with convoluted explanations for why doing so is unnecessary and counterproductive. The assertion is basically that capitalism is so terrible that once we get rid of it, things will _obviously_ be much better.

Apologists have tried to fill in the gaps with a utopian vision [1] where we maintain the a similar standard of living but without wasting energy on conspicuous consumption. I agree that developing a culture that avoids conspicuous consumption would probably mean we could get by with less work. However there would still be enough work leftover to either require capitalism, or socialist style forced labor, both of which situationists explicitly reject.

[1] see [https://en.wikipedia.org/wiki/Hans_Widmer](https://en.wikipedia.org/wiki/Hans_Widmer)

</comment><comment><USER6> Hmm, [https://www.youtube.com/watch?v=NhheiPTdZCw](https://www.youtube.com/watch?v=NhheiPTdZCw) anyone?

(Finish the video before replying.)

</comment><comment><USER7> I absolutely loved this when I first read it in the early 90s, it's amazing how it seems _more_ accurate as time has gone on.

Don't dream here, be-he-here ... ;-)

<comment><USER8> Is that a modified Rocky Horror reference?

</comment></comment><comment><USER9> for an excellent and knowledgable look at the si, its origins and legacy read The Beach Beneath the Street: The Everyday Life and Glorious Times of the Situationist International by McKenzie Wark [1], author of the si influenced look at technology 2004's A Hacker Manifesto

[https://www.versobooks.com/books/1869-the-beach-beneath- the-...](https://www.versobooks.com/books/1869-the-beach-beneath-the-street)

</comment><comment><USER10> Thanks :)

This reminded me of the Spectacular Time by Larry Law

</comment><comment> nils-weiher Jesus is the only way to a real life

</comment><comment><USER11> I feel like this XKCD strip is basically Debord's argument in modern terms and a response it: [https://xkcd.com/1314/](https://xkcd.com/1314/)

And yes in a sense it's strangely prophetic that we now live in a world where we feel compelled to do things like tweet and instagram and facebook and etc etc etc.... That we seek out experiences to make a spectacle of them, and that we see the world that way(e.g.): [https://xkcd.com/77/](https://xkcd.com/77/)

But what I really take issue with is this need to blame it on capitalism, and not human nature.

We constantly strive to compete with each other, socially - and now we have means to do so on a much broader scale before. I don't think capitalism has some sort of "dark agenda" that pushes this forward. Advertisers use this against us, Facebook taps into it, etc, but it's human nature; one of those many cognitive biases that causes us to seek what we think we want, not what will actually make us happy.

Pre historic people were almost certainly showing off to other pre historic people - and others were watching. Let's not pretend it's just because of Capitalism we feel compelled to do this.

<comment><USER12> I do believe that debord was aiming for more than shallow moralizing about the evils of capitalism. Marx certainly considered capitalism a step forward. But if we use the word capitalism at all, we are assuming that it constitutes a useful independent variable for analysis; cultural, sociological, or otherwise. Debord does a deft job at this analysis, irreducible to the average complaints about Facebook or twitter. For example, how might Debord's critiques be applied to the phenomena constituting the Arab Spring? Think about what sparked the Arab spring. Think about the Egyptian revolutionaries' adoption of the Guy Fawkes mask. Juxtapose those 'spectacles' against Debord's observation that 'the "society of the spectacle" offers false models of revolution.'

imho, these illustrations are probably counter-productive insofar as they turn debord into a parody of himself, a straw man to knock over, rather than what he was: a truly dangerous thinker.</comment></comment>


SF Bay Area Hackathon / Networking Event - <AUTHOR>

Student looking to get feet wet learning more about programming on an awesome product this summer by working with a startup. I am experienced with Java and C++ Code as well as HTML and some light JS. <comment><AUTHOR> Project Code here: <https://github.com/jweems/Warehaus/tree/master/order></comment>


Distressed ravens show empathy - <AUTHOR> https://medium.com/@GrrlScientist/distressed-ravens-show-empathy-is-for-the-birds-too-grrlscientist-248338c1809e#.jdtlkzqy9 <comment><USER2> I have a strange love for birds. People often laugh at the endearment I have for birds like they have for dogs or cats. But when asked why I like birds so much, I can answer this very succinctly: there is very few other species that can fly, walk or swim all at once like some birds can. Nor a group of species so diverse, colorful, capable and intelligent. With such an interesting evolution sharing an ancestry with the once terrifying dinosaurs that roamed the world now so commonplace and taken for granted.

Just the intelligence of birds alone is fascinating with birds that can learn human speech and understand its context, solve puzzles, solve puzzles in teams, understand the concept of self, learn human faces, give gifts, have theory of mind, perform ritual and now show empathy. All packed into a tiny "bird brain." They are truly marvelous creatures.

<comment><USER3> All that, and they're delicious, too.

<comment><USER4> Your comment reminds me of a story in a book called Shantaram. Guy feeds a mouse visiting his cell every now and then. Before being relocated this prisoner tells the new inmate to take good care of Mickey. Next thing he knows, his little friend is on display with two pencils piercing his body. Crucified.

<comment><USER5> :(

</comment></comment></comment></comment><comment><USER6> Corvids are bloody amazing. Arguably, they also have language.[0] Nightingales too.[1] But bird languages are very different from mammalian ones. Maybe more like neuronal activity directly mapped to sound.

[0] Mates (2016) Locally Periodic Signal Estimation and Pitch Detection, and Acoustic Communication in American and Northwestern Crows (PhD thesis) [https://dlib.lib.washington.edu/researchworks/handle/1773/35...](https://dlib.lib.washington.edu/researchworks/handle/1773/35633)

[1] Weiss et al. (2014) The use of network analysis to study complex animal communication systems: a study on nightingale song [http://rspb.royalsocietypublishing.org/content/281/1785/2014...](http://rspb.royalsocietypublishing.org/content/281/1785/20140460)

<comment><USER7> Hey mirimir, it's awesome to see posts from someone else that thinks birds are cool. I hadn't seen those papers before--that Weiss study looks like it's worth reading in depth.

Just wanted to say that most people who study birdsong and bird calls wouldn't describe it as a "language"\--as far as we know, humans are the only animals with language, unless you believe that one guy who wrote the article about dolphins. But you're right that birdsong is a lot more like language than most other animal vocalizations; often the songs have "syntax" (like in the Weiss article you posted) where the structure follows rules. But the syntax doesn't convey meaning in the same way it does in our language. No bird has ever written a philosophy book, right? They're just communicating stuff like "this is my tree and I'm so manly I can sing in this tree for hours in the middle of the night".

Along the same lines, I think birds are pretty close to us evolutionarily speaking. You're not the first person I've heard compare them to aliens, but their brains are actually a lot like ours. Here's a grumpy blog post I wrote about it if you want to have a look: [http://www.theneuroethicsblog.com/2012/09/snakes-on-brain- or...](http://www.theneuroethicsblog.com/2012/09/snakes-on-brain-or-why-care- about.html)

As the article above demonstrates, along with many other behavioral studies like those you mentioned, birds behave a lot like we do. So: their brains are a lot like ours and their behaviors are a lot like ours. This is why we should study them even more, if we want to understand intelligence.

Think of it this way: you're an alien and you want to figure out how a computer works. If you could only carry out your experiments with MacBooks, it would probably take you longer than if you could also use e.g. Lenovo ThinkPads (just trying to draw an analogy, I'm not throwing shade at Mac or anything).

I hope that wasn't way too "tl;dr" but I just wanted to build a little bit on your comments.

<comment><USER6> > Just wanted to say that most people who study birdsong and bird calls > wouldn't describe it as a "language" ...

Yes, I get that. And maybe they're looking too hard for something like human language. Maybe it's more like communication protocol. I mean, if you down- shifted Ethernet traffic and listened to it, would it sound like language?

> They're just communicating stuff like "this is my tree and I'm so manly I > can sing in this tree for hours in the middle of the night".

Well, if you didn't know any human languages, and your system clock were really slow, maybe English would sound like that to you. So if I'm out dancing, I'm wasted, and I'm using earplugs, casual conversation __does __sound a lot like that to me ;)

> Along the same lines, I think birds are pretty close to us evolutionarily > speaking. You're not the first person I've heard compare them to aliens, but > their brains are actually a lot like ours.

Yes, I get that too. The basic layout is the same. But when reptiles and mammals diverged, their brains were much simpler, and arguably neither line was very intelligent. So brains and intelligence have evolved in parallel in birds and mammals. And maybe structural similarities just reflect the shared lineage.

</comment></comment></comment><comment><USER8> This is compassion / responsiveness, not empathy.

Empathy = mental simulating of how an action will affect another.

Compassion = a desire to help another.

Responsiveness = the ability to recognize changes in someone's emotional state, and then develop and engage in a course of action based upon that change.

More info: Heinz Kohut - Reflections on Empathy [https://www.youtube.com/watch?v=ZQ6Y3hoKI8U](https://www.youtube.com/watch?v=ZQ6Y3hoKI8U)

</comment><comment><USER9> I strongly recommend the book Ravens in Winter for anyone interested in this topic.</comment>


Jeep Bluetooth Radio Hack for a TJ Jeep - <AUTHOR> http://longhornengineer.com/2017/02/06/jeep-bluetooth-radio-hack-for-a-tj-jeep/ <comment><USER2> This is a great hack! For those of us that don't have time but want a similar result, 1FactoryRadio sells a lot of OEM radios with bluetooth upgrades.

[http://www.1factoryradio.com/1984-2002-ras-chrysler-jeep- dod...](http://www.1factoryradio.com/1984-2002-ras-chrysler-jeep-dodge-car- truck-w-bluetooth-music-cs-am-fm-radio/)</comment>


IList, IComparable, ICarumba - <AUTHOR> http://blog.8thlight.com/micah-martin/2006/09/07/ilist-icomparable-icarumba.html

<comment><USER3> Just to get this out the way: In his example, his resulting command should be (if hes such a stickler for naming conventions) CommandBase (or BaseCommand).

Prefixing Interfaces with an I is not a mistake; Using an Adapter Pattern when one is not the right choice, is the mistake.

"I can't think of any logic to put in the base class and the Dependancy Inversion Principles says interfaces are preferred. So I'll make it an interface and follow the IConvention naming it ICommand."

But then he does think of logic that he wants in a base class, as he goes about developing it. Also, hes following DI principles without first thinking about how those principles apply. It feels as though hes just putting fingers to keyboard without putting thought towards the design itself.

We've all run into this scenario, and yes it can be a pain in the ass - <USER2> why you should stop and think before writing any code and make sure the choices you're making now, are the choices you'll want to live with later.

As for the hassle of having to change things... I don't think theres a developer worth his salt who hasn't gone through that scenario a dozen times over, and probably will go through it another dozen times atleast. Things always change partways through development, and you realize you based later work on an earlier assumption that did not hold up. It's part of the process.

tldr; Interfaces begin with an I, Base classes should be labeled as such</comment>


"The switch had two positions... 'magic' and 'more magic'. " - <AUTHOR> http://catb.org/jargon/html/magic-story.html <comment><USER2> Hilarious story. And Gerry Sussman swears it's completely true.

edit: GLS = Gerry Sussman, creator of Scheme, co-author of SICP, and MIT professor.

<comment><USER3> Hand in your geek black belt when you leave.

This GLS is Guy L. Steele (also a creator of Scheme). Gery Sussman is Gerald J. Sussman.

You may keep the geek card.

<comment><USER2> Oh, shoot. Too true. Gerry definitely recounted this story to me, although I can't recall now whether he claimed he was involved or not.

I am Aston's total lack of geektitude.

<comment><USER4> "Gerry definitely recounted this story to me"

\- so your geek card is lifetime membership eh ?? ;)

</comment></comment></comment></comment><comment><USER5> From mccoyn's Reddit comment:

"The isolated grounds was what I thought it was while reading the story.

I bet when this was installed, there was a problem with the sound buzzing. Isolating the grounds would make it go away, like magic. But reconnecting the grounds while the computer was on would create a sudden ground change that might crash the computer, more magic."

</comment><comment><USER6> Being the curious type, I don't think I would have let such an interesting switch sit idly by for a year like this guy did. Why didn't he investigate it further after first making the discovery?

Such a great story though.

<comment><USER7> Triage (though I bet it kept popping into his head at inopportune times).

</comment></comment><comment><USER8> At first glance i thought this was about VIM magic...

<http://www.vim.org/htmldoc/pattern.html#/magic></comment>


Show HN: The Elo rating system applied to photo, a new way to do photo contests - <AUTHOR> http://www.pixide.org <comment><USER2> Similar to this post from 5 days ago I think?

[https://news.ycombinator.com/item?id=12878336](https://news.ycombinator.com/item?id=12878336)</comment>


Ask HN: What are your dreams? - <AUTHOR> A bit lost in life and very curious to hear about what others dreams are and what they strive for these days (success, family, peace, etc). Thanks! <comment><USER2> My dreams have morphed over time as my circumstances have evolved. These days my dream is to have more serenity and to strive for contentment. I hope that a bit that mindset inspires my immediate family and friends.

In our culture the trap of chasing fame, success and ever more money is a never ending vicious circle. There always will be those who have more disposable income than us. That is just the way it is. So as hard as it was for me to accept that earning more only works to a point, I can finally say that I am able to keep my ego in check and live within my means. I haven't stopped in my efforts to deliver ever greater value, it simply means that I am content with the process of learning and gaining the expertise that in turn is better remunerated.</comment>


Show HN: Writing for Software Developers - <AUTHOR> https://philipkiely.com/wfsd/ <comment><USER2> Something, something, "Build your brand by writing."

Something, something, "Communicating with words is as important as communicating with code in a team setting."

But moving up Maslow's Hierarchy, writing about ideas you want to personally explore (a'la pg) is exactly how you force yourself to think those ideas through. It's valuable in its own right, whether people read them and shower you with the "three Ps" or not.

So write, and write often. Do not spend great amounts of time carefully choosing what yo write. Just write, and incorporate feedback into your work. Revise endlessly, much as one refactors software endlessly.

Do not write about things that don't interest you just because you get more up-boats for "How to use Flibbitz in a managed K8s container with XML configuration," unless Flibbitz personally interests you.

Write about what tickles your brain, and keep at it incessantly. What little (and it's not much in the scale of things) success I've had writing essays and books can all be traced back to writing about things I personally care about.

<comment><USER3> What are the three P's?

<comment><USER4> Yes, which?

[https://www.google.com/search?q=the+three+p%27s+of+writing&r...](https://www.google.com/search?q=the+three+p%27s+of+writing&rlz=1CDGOYI_enUS590US590&oq=the+three+ps+of+wr&aqs=chrome.1.69i57j0.15813j0j7&hl=en- US&sourceid=chrome-mobile&ie=UTF-8)

<comment><USER5> Haha, yeah! I'm having a blast looking at the search results...

Planet, People, Profit. People, process, product People, Process, Performance Management Personalization, Permanence, Pervasiveness Polydipsia, Polyuria, Polyphagia Product, Price, Place and Promotion Preserve / Prevent / Promote Prediction, Protection, Preparation Purpose, Process, Product Protect, Procreate, Provide. Protect, Provide, Profess Pee, Paper, Poo Purity, Patience, Perseverance Provision, Protection, Participation Presence, Promises, Power Of God Piracy, Portability, Profitability Presentation, Planning, Processing

But still no idea... OP, which is it?

<comment><USER6> Practice, Practice, and Practice, I think.

</comment></comment></comment></comment></comment><comment><USER7> Hey Philip,

really great to see you run with this! I know you mentioned to me that this repository[0] where I try to keep a short list of publishers that pay for this kind of writing was an early inspiration and I'm still hoping to grow that list a lot after June when I spend more time focusing on technical writing too.

As others have said, that's a really great looking list of interviewees. My only question is where this actually fits in to the rest of the book? It's not that clear from the Table of Contents. Could you comment on what percentage of the book is interviews, compared to writing advice?

[0] [https://github.com/sixhobbits/technical- writing/blob/master/...](https://github.com/sixhobbits/technical- writing/blob/master/write-for-us.md)

<comment><AUTHOR> The table of contents outlines the book, which is about 30,000 of my words and 28,000 words of interview excerpts. Then, the appendix contains the full transcripts of all 11 interviews, 45,000 words (which does overlap with the interview excerpt word count from earlier).

</comment></comment><comment><USER8> Writing seems to be the single best thing you can do as a professional, in other areas as well. I know people who were hired because of a single Medium article that got a lot of traction (and made that person sort of a reference in the industry). What do you think about videos? I see lifestyle channels for devs and tech workers taking off but do you think there's a chance the same might happen for more technical channels as well? I find technical videos (like talks or tutorials) to be far more engaging than articles but I'm afraid I'm part of a very small minority.

<comment><USER9> Forced pace is a huge problem for videos. If you encounter a paragraph that you already know all about in an article, you can skip it. If you encounter a section of a video like that, it's harder to skip so I'll end up trying to sit through it, getting bored, then distracted, and then who knows what I'll miss.

The 'killer feature' of writing is the ability to self-pace. To me that makes them orders of magnitude more effective as a means of communicating ideas.

<comment><USER10> Written materials can also be used as reference much more easily. You can't ctr+f a video to look for the specific nugget of information you need.

</comment></comment></comment><comment><AUTHOR> Hi HN,

I have been working on Writing for Software Developers for six months and I'm so excited to finally launch it! I set myself the goal of launching before my (now virtual) university graduation and I'm proud to have hit that goal by six days.

Right now the book is 12 dollars off (launch pricing) and you can read chapter 1 for free at [https://philipkiely.com/assets/files/wfsd_chapter1_sample.pd...](https://philipkiely.com/assets/files/wfsd_chapter1_sample.pdf)

The book features interviews with 11 people, including @patio11 (Patrick McKenzie), who had the following to say about Hacker News. This quote is from Chapter 15, which covers promoting your work:

"At one point I was the number two user on Hacker News. I might have slipped down to number three...I think that people underrate Hacker News massively. I think there is a meme in the community that Hacker News threads are populated by toxic commenters and that it is a ceaselessly negative place such that the world would be better without Hacker News in it. I think unequivocally Hacker News is an extraordinary venue for value creation throughout the world, largely by bringing together technologists who would not have otherwise met each other. There are people who make lifelong relationships from that site. I met my former co-founders there; Thomas Ptacek is one of my best friends for life; and it feels extremely unlikely that I would have my current job but for Hacker News. I think you could say that for many people who are nowhere close to being quote-unquote on the leaderboard.

Hacker News helps disseminate ideas, like my ideas on sales and engineering career optimization and the body of practice that is dealing with venture funding, both from the startup side of the fence and from the investor side of the fence. These things would be difficult to get access to unless you had a high-quality social network that already had an expert about them in it. It is basically one step short of miraculous that you could find George Grellas, who is an expert Silicon Valley lawyer practicing for thirty years, and have George Grellas patiently walk you through the impacts of the changes in the independent contractor classification in the wake of Microsoft abuses in the 1990s. By the way, George Grellas was a practicing lawyer for Microsoft. That I could read that as a person who was newly an independent contractor working in central Japan in the late 2000s/early 2010s is one step short of miraculous.

I'll acknowledge that there are certain threads that are generally low quality, mostly things that are removed from the core interests of the technology industry. It's not a particularly great watering hole to talk about politics, for example. But, for the things it does well, Hacker News does them better than plausibly any place in the world...I wish more people who could get value out of Hacker News were active there."

Thanks to all eleven people who provided interviews for the book. Thank you for checking out my project! I'll be around to answer any questions.

<comment><USER11> thats awesome, congrats on launching!

as a fellow soon-to-be first time gumroad author - what was your tech stack for writing and publishing? i tried Ulysses and Scrivener and Leanpub and wasn't happy with any of them. looking to go from markdown to PDF/Epub/Mobi with good control over table of contents, footnotes, and linking between chapters.

i have a list of alternatives to explore.. just wondering what worked for you?

also - how did you mock up that book cover art on the landing page?

<comment><AUTHOR> I wrote in Markdown as well. I turned that into a microsoft word doc that I formatted then created the PDF. I used pandoc to create the EPUB, then calibre to make the MOBI from the EPUB. I tried to use pandoc to make the PDF but it did not go well, word was surprisingly easy to use for that.

The cover art was created by a friend of mine from college. I used [https://diybookcovers.com/3Dmockups/](https://diybookcovers.com/3Dmockups/) to make the image on the landing page.

</comment></comment></comment><comment><USER12> Hey Philip,

Congrats on the launch of your new book. Will probably be checking out the first chapter after work.

As I've been hanging out more online, I've realized the increasing importance of writing. Even as a developer, writing seems to be a differentiating factor.

Other than your book, what are some other resources that you relied on in order to build up your skills as a writer? Or did you just rely on reading and emulating people's works.

<comment><AUTHOR> Hey!

Thanks for checking it out. One of the reasons I wrote this book was because there were so few resources for me starting out. I mostly relied on my experience as a journalist with my college's newspaper plus the training we got in CS classes on writing good documentation.

</comment></comment><comment><USER13> I didn't carefully read the whole article, scanned through it and read the comments. Sorry if I missed the answer.

Is this an intro / "how to" book about technical writting, or a book with a series of interviews with good writers. In the same genre as "Founders at work"?

<comment><AUTHOR> Both! The first act is step-by-step on how to write technical tutorials. The second act is examples of that process. The third act is about the business of writing. All of that is a book that I wrote, but does rely heavily on block quotes from interview subjects, plus 30,000 of my own words :)

Then, the appendix contains all of the interview transcripts, totaling 45,000 words.

<comment><USER13> Thank you for clarifying

</comment></comment></comment><comment><USER14> Keep writing!! I've personally started my journey into content creation as a software engineer just this year! To be more focused, I found a growing passion for Design Patterns and started writing my learning about them. Hope to share the same passion with you! Keep going pal! Would love to hear some tips from you! Here's my site, and hope it encourages you all!

[https://blog.phuaxueyong.com/post/2020-05-03-3-more-cloud- de...](https://blog.phuaxueyong.com/post/2020-05-03-3-more-cloud-design- pattern)

<comment><USER15> This is so cool! I've just started myself (still putting together my blog). I really want to learn functional programming, maybe writing about it would be a good way to start on my journey too!

<comment><USER14> I'm definitely supporting you on this idea! I've learnt so much through "Learn as you teach" methodology. Do share your site once you have started it (:

Oh and personally, as a way to learn my frontend development skill, I also created my blog from scratch through a frontend framework! You can consider that as well

</comment></comment></comment><comment><USER16> Wowowow! I've only read the FREE first chapter so far, but honestly, I'm pretty hooked. If the rest of the book is as useful and well-written as the first chapter (which I suspect it is) then the price is a steal. Following this book's advice in writing your own tech articles will earn you your money back from buying it within a couple of weeks.

Well done philipkiely!

</comment><comment><USER17> The purchase link isn't working for me. I get a certificate error.

Edit: It appears that by connecting via my VPN, it gives me a certificate error when the link redirects via gum.co... goes away when I disable it.

Seems worrying...

<comment><AUTHOR> I'm sorry you experienced difficulty purchasing it. If you are still having issues you can email me firstname.lastname@example.com I'll see what I can do.

</comment></comment><comment><USER18> The website says "Earn money, make connections, and get noticed.The rates I earn from writing, usually $250 to $500 per article, add up." What are some ways to do that exactly?

</comment><comment><USER19> I wanted to just automatically say booring! But then I clicked the link and I love it, thank you!

It's short, simple and to the point. My personal holy trinity. See you around)

<comment><USER19> Downvotes? Love it)

<comment><USER20> Please don't break the site guidelines: [https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html).

</comment></comment></comment><comment><USER21> Hey Philip, this is great. I'm curious, what tools did you use to produce the book?

<comment><AUTHOR> I wrote in Markdown. I turned that into a microsoft word doc that I formatted then created the PDF. I used pandoc to create the EPUB, then calibre to make the MOBI from the EPUB. I tried to use pandoc to make the PDF but it did not go well, word was surprisingly easy to use for that. The cover art was created by a friend of mine from college. I used [https://diybookcovers.com/3Dmockups/](https://diybookcovers.com/3Dmockups/) to make the image on the landing page.

</comment></comment><comment><USER22> Amazing! Just purchased it. Thanks Phillip.

Question on the side: does anybody know of a good app that could "read" the epub for me? I feel like I could listen to these interviews while doing chores at home.

<comment><USER23> This app @Voice Aloud Reader is great for all kinds of docs. You can even long press paras in Chrome Android and in the long-press menu there'll be an option for Reading it Aloud.

epub,PDF etc. supported too, all details in app description.

[https://play.google.com/store/apps/details?id=com.hyperionic...](https://play.google.com/store/apps/details?id=com.hyperionics.avar&hl=en_IN)

</comment></comment><comment><USER24> "Throughout college, also I have sought physical engagement outside of martial arts, taking up climbing and lifting." [0]

Not to be harsh, but I wouldn't buy a book from someone writing about writing that writes awkward/grammatically incorrect sentences like the one above. Also your background doesn't "checkout". You're writing about writing for software engineers, but how long have you been a software engineer? 3 years? 5 years?

Again, my comments are a little harsh, but this has to stop with people filling the internet with useless content, making it harder and harder to find good content.

[0] ([https://philipkiely.com/essays/black-belt-white- belt.html](https://philipkiely.com/essays/black-belt-white-belt.html))

<comment><AUTHOR> Thank you for taking the time to let me know how to improve my writing. Would you be so kind as to provide editorial feedback on other pages on my site? I look forward to incorporating your suggestions next time I revise my work.</comment></comment>


Peter Schiff on Abolishing Student Loans to Make Colleges Affordable  - <AUTHOR> http://schiffforsenate.com/?q=media/how-government-programs-drive-college-tuitions <comment><USER2> This is where good voice recognition would come in handy. I'm interested in the topic but I'm not the least bit interested in watching a 10 minute video; I'd much rather have the text.

<comment><USER3> if your interest is more than passing you may want to delve into the more general principle of supply/demand distortion.

<http://en.wikipedia.org/wiki/Subsidy>

<comment><USER4> Come on, there is more to federal college loans than supply/demand distortion. Easily accessible student loans have much further reaching societal and economic implications than "inflating tuition costs", and those should play into the calculus of whether government-sponsored student loans are worthwhile.

This guy's example of Yale's 42 year stint of $33 tuition and so forth is pretty poor, too.

"What has changed? Back in the 1800's there was no Department of Education, no government guaranteed student loans..."

Gee, mainly that there were only a few sons of aristocrats going to the few universities in the country, and all they did there was read Great Books. Today universities are centers for research and have much broader educational missions.

Guess what? Tuition isn't the primary source of income for universities (at least not mine, a Tier I research university). It's /research grants/. Tuition allows you to pay for high-caliber faculty that can bring in huge grants. Schiff misses this point /entirely/. We keep raising tuition so we can offer salaries competitive with our peer institutions and keep getting top-notch faculty (as an out-of-state student at a public university, I'm quite cognizant of these increases).

"If the government didn't guarantee student loans, no one would give them, it'd be too risky."

Completely false. Student loans are some of the least-risky loans to give: the expected income of future doctors, engineers, etc easily justifies the risk of a loan. That's why there is a thriving private market for non-government guaranteed student loans.

<comment><USER5> Then maybe universities and research centers should be separated. My tuition should be applied towards the professors who will be teaching me. I'm going there for my own personal progress. If I want to go to a university that's doing the research then I would be willing to pay the extra tuition to go there.

<comment><USER6> University of Phoenix is open 24 hours a day, go for it.

<comment><USER7> You're probably being facetious, but I suspect that University of Phoenix and similar programs are, as an idea, higher education done right. My bet is the for-profit colleges will give establishment education a serious run for their money in coming years.

</comment></comment></comment></comment></comment></comment><comment><USER8> Reading people comments a lot of people says this man has no backing information, but I got a similar experience in Spain.

What happened in USA with education is what happened in Spain with homes:

Here home prices exploded, and remain high today, a 30m^2 flat(322square feet)in Madrid sells by 180keuros.

Normal flats cost no less than 300keuros, the interesting thing is that the prices started rising just when Aznar(he wanted to help as the USA gob) removed the limitations in loans years, and backing money you need for making a loan.

Normal loans went from 8 years to 10->20->30->40 years. Prices went up because someone with less money you have will pay more with virtual money from a 40 years loan he got from the bank without putting a cent.

When people could not pay the loan, they refinanced the loans with more years. That worked until now, things are not going well.

In Spain it costs over 1.000 euros/year to get a quality degree in Medicine or Engineering from a public university, this is 10% of the real cost of your education (over 10.000euros/year) for the state.

I regularly see MIT and Stanford lectures on Internet. They are not better that the education I received, but is way way more expensive.

</comment><comment><USER9> His main premise, that government student loans are co-signed by the government and students resulting in the distortion of demand, is spot on. Demand = Need + Purchasing Power. With the Purchasing Power of the government at students' backs, schools would see that they needed to raise prices to keep supply at the same level.

<comment><USER3> there are other issues at play too though. schools don't currently maintain a supply and demand equilibrium. they under price and then use rationing. private schools _could_ simply raise prices until number of applicants is approximately the number of positions available. but of course there is a prestige economy in place that interacts with the money economy.

<comment><USER6> Great comment - the prestige economy does seem to have more of an effect on college price than any traditional supply/demand curves. We'd all be wise to recognize that.

</comment></comment></comment><comment><USER3> it's fairly depressing to me how no one seems to grasp the concept of _price_. money isn't magical, it's a way of accounting for different people's preferences WRT the resources they have available to them. trying to fix a price by decree is a fundamentally stupid notion.

<comment><USER10> On the other hand, college is an industry, like healthcare, where demand is highly inelastic due to incentives involving third-party payments. It is also an industry, like healthcare, where price inflation has far outstripped inflation in general.

Is there an example of an industry where government-created demand accounts for a major portion of consumer spending and there is not rampant price inflation? I am open to the possibility that there could be other causal factors here, but the a priori logic is pretty compelling.

<comment><USER3> I'm sort of confused by your use of the term "on the other hand". I'm in complete agreement with Schiff (and economists in general) when talking about the effects of price distortions from tax/subsidy (two sides of the same coin).

</comment></comment></comment><comment><USER6> So the argument is that if they eliminated student loans, colleges would have to cut prices in order to maintain demand, right?

Isn't that a little risky, with the chance of pricing out the lower 1/2 of income earners in the US?

What if, as a compromise, they just eliminated college loans for those with family incomes over a certain level? Then you could get the economic effects without consigning a third of a generation to "sorry you were born in the wrong place kid, no college for you".

If we want to eliminate economic distortions, we should go after the farm subsidies, of course. But those are even more of a political minefield than this idea.

<comment><USER11> Cutting student aid would lead to lower tuition prices. More importantly, it would probably lead to completely different kinds of educational institutions. If students and their families had to more directly bear the costs of their educations, they would be more motivated to seek the best values.

By making college education artifically cheap, government discourages competition and innovation.

The best example are primary and secondary public schools which typically are completely free to a student. A private school, even if it delivers a much better educational value, has a hard time competing because if it charges anything, it's already more than what the student has to pay to attend public school.

The only way for private schools to compete in such an environment is to offer something the public school doesn't offer. A private school cannot compete by offering the same thing the public school offers but just at a lower price.

<comment><USER6> RE: private schools, assuming you're referring to primary/secondary school as opposed to universities (where private schools thrive).. do you know how Charter Schools work? Ever seen some of the state laws regarding those? In many states, charter schools actually get to operate with a financial advantage over the public schools. I used to be in public office and watched a Charter school take advantage of the laws to siphon a ton of resources from our public school (which was already cutting teacher headcount year over year because of healthcare costs), and then to add insult to injury we had to pay to _bus_ the kids there.

Anyways, back to the larger point.. I put a very high value on the idea that anyone in America, if they have a little talent and put in some work, can get a college education and make their own destiny.

That particular dynamic has _way_ bigger implications for American competitiveness than moderate price distortions in university tuition -- I mean, if we can fix those, great, but let's not throw out the baby with the bathwater, and burn our house down afterwards for good measure.

</comment></comment></comment><comment><USER12> What's worse is the way for-profit colleges are abusing the system and taking federal loan dollars for subpar educations.

Their default rates are appalling, but that's to be expected when you charge $40k for a two year degree in Fashion Design. This should have been reigned in years ago.

[http://www.usatoday.com/news/education/2009-12-14-student- lo...](http://www.usatoday.com/news/education/2009-12-14-student-loan- default_N.htm)

<http://federalstudentaid.ed.gov/datacenter/cohort.html>

<comment><USER13> Plenty of "non-profit" colleges also take federal money and provide subpar educations.

The only real difference is that the non-profit colleges funnel money to employees rather than to shareholders.

<comment><USER12> Yet non-profit colleges have a repayment rate of 95.3% while for-profit colleges sink to 83.3%. So it's not the 'only real difference'. But I'll give you that non-profits are not entirely blame free.

<comment><USER13> The better colleges are non-profit, which makes the non-profit group look better than the for-profit one. The higher quality students only go to non- profit colleges, and thereby lower the default rate of non-profits.

If a Devry student would not be accepted by Princeton (but would be accepted by Bumfuck U), it's unfair to compare Devry to the average of Princeton and Bumfuck. One should simply compare default rates of Devry and Bumfuck.

</comment></comment></comment></comment><comment><USER14> Interesting idea.

Related: If the government announced that in 10 years the minimum deductible for health insurance would be locked at no less than $20,000, what would that do for health care costs? In other words, if the only legal health insurance was catastrophic, would the health care industry be forced to become affordable?

</comment><comment><USER15> You can't simultaneously believe that the free market tends towards a stable and optimal equilibrium, and also believe that a government subsidy will wildly and erratically impact the market. These two beliefs are mathematically incompatible.

<comment><USER16> Well obviously you _can_ believe that because there are people who _do_ believe it.

As for your mathematically incompatible assertion, the way that the free market tends towards an equilibrium is because of feedback mechanisms that cause behavior change. However in the presence of a government subsidy the price signal of increased tuition does not necessarily cause decreased consumption. This breaks the feedback loop that is supposed to result in equilibrium.

This is a fundamental change in the mathematics of the situation, and can justify the belief that the market behaves in radically different ways in the presence of a government subsidy.

<comment><USER15> If that were the nature of the subsidy, then yes, that would break the tendency to return to equilibrium in the face of altered data. However, only a fraction of tuition is paid by the government, so there is still a significant price signal to the consumer.

</comment></comment></comment><comment><USER16> Here is my modest proposal.

1\. Fix student loans to give sane amounts of support to students. (Cost of living has gone up in the last few decades, but the estimated cost of living for your parents they use in the formulas haven't been adjusted. That's ridiculous.)

2\. Only offer government supported student loans to students whose university tuition levels are in the bottom 70% of accredited universities.

The first change fixes the problem where a lot of people who need support don't get it. The second change gives universities an incentive to not just automatically bump tuition 10%/year "because everyone else is doing it".

</comment><comment><USER17> He makes a valid point and personally I think he is probably at least partly correct, and it's something that needs to be debated. However his argument is flawed because he focuses on the correlation between the government programs and tuition rates without giving any evidence of causation. His entire argument is based on "This then That, so This caused That" which is rarely the case. I can't think of any good examples of this so if someone else understands what I'm saying please help me out.

</comment><comment><USER18> "Well, the world needs ditch diggers, too."</comment>


What is the business model for DuckDuckGo? (2017) - <AUTHOR> https://spreadprivacy.com/duckduckgo-revenue-model/ <comment><USER2> DuckDuckGo has 50-100 employees, according to CrunchBase, while Google has over 100,000 employees. The exact numbers are irrelevant, but Google has around three orders of magnitude more employees than DDG.

a) DDG needs about 1/1000th the revenue of Google to survive at its current scale.

b) Software quality does not scale linearly with manpower. A 50-100 person team can do /almost/ as well as a 10,000 person team in the same domain. You spend a lot to go from 90% to 99% or 99% to 99.9%.

c) Unsurprisingly, I find DDG's search results good enough. I rarely go to Google, and only when DDG fails me.

d) If DDG can't target advertising as well as Google, they'll still have enough revenue to survive at their current level of quality. Their CPM is likely lower than Google's with less data and optimization, but I'd guess at worst 2x lower, not 1000x lower.

I'm glad they're doing well. On paper, it seems like a sound business model, and it's a heck of a lot more customer-friendly.

Curiously, I'd say 90% of the time I use Google is for math. If I'm typing a complex math expression (especially with units), Google tends to do much better than DDG.

<comment><USER3> When I switched to DDG, I was happier with it than with Google. To me, DDG feels like Google felt ten years ago, before it was infested with "AI." Today, Google search results look more like the invasive shopping mall ads in Minority Report, tailored to what it thinks I'll click on, rather than relevant results based on my query. I only rarely (~once a month) have to use Google search to find something that DDG just cannot, and usually this is some highly technical topic. (I did it yesterday because I couldn't find information about a strange, fatal regex error in Firefox - seriously, one regex that Firefox can't understand prevents script execution, even when wrapped in exception handlers.)

I think Google has hit some sort of inflection point where their further attempts to improve search have made it a less useful experience and are mainly in service of their dual corporate mandate to put al ads in front of all eyeballs and control all access to all information.

<comment><USER4> > When I switched to DDG, I was happier with it than with Google.

I regularly do !g, but I think the thing that is reducing that fastest is the declining quality of Google results. If I search for 'x, y and z', I then see a load of results with small text underneath saying 'include z?'. It drives me demented, why wouldn't I want to include my search terms?

<comment><USER5> > It drives me demented, why wouldn't I want to include my search terms?

Well, I think the answer to your question is Google is trying to be friendly to the layman. It's not hard to think of examples where the user inputs a something into the search box that a CS major knows will not result in the implied wants of the user. Google tries to do a lot of human to machine translation work, and I think you're seeing the result of it. That being said, you can use quotations to explicitly include terms.

<comment><USER6> This works for the masses, but in my experience when machines force a use case, it causes more headache than not cause I'm not like the masses.

A barely related side rant: my microwave decided it wants to error out when it thinks you're trying to microwave air. i have a bowl in there. stop telling me to open the door to put something in. Whatever engineer at GE thought they would be smarter than the user, you're one of the many reasons your company is going out of business.

<comment><USER7> I think that it's O.K. when a user interface which issues a warning when it detects a dangerous condition.

The problem is when the warning is a "stop" and it doesn't have an override "I know what I'm doing".

</comment></comment></comment></comment></comment></comment><comment><USER8> I like DuckDuckGo, but it's much worse than Google at localisation. It's annoying to have to search, then repeat the search with "UK" on the end to make it realise I'm not interested in buying e.g. building supplies or coffee or beer from US-based businesses.

I get that it has no concept of who I am when I search, but some kind of localisation based on IP address could maybe help?

Also, it feels a little disingenuous to say Google is not a search company, it's an advertising company. DDG's revenue is also built on ads. Is it fair to say they are also not an advertising company, not a search company?

<comment><USER9> There is a slider that allows you to put it on or off.

If your IP is in the UK, the slider will say "United Kingdom on/off" or something like that.

<comment><USER10> Right, but it's kinda annoying having to turn it on and off, as it remembers and I keep forgetting what I used last.

If I'm searching for programming stuff or similar, I want it off. If I'm searching for shops that sells bricks for my new wall, I want local results.

I've also noticed that even with it on, the "local" indexing is distinctly significantly worse than Google. That is, Google is much, much better at finding relevant pages which may contain variations on my search term when using my native language.

I guess that's just a question of resources, which is fair enough.

<comment><USER11> > If I'm searching for programming stuff or similar, I want it off. If I'm > searching for shops that sells bricks for my new wall, I want local results.

In all fairness, how exactly do you expect DDG to know what you want from this, without doing some sketchy privacy invasive scanning of your search? If you really want a company to scan your every search and then tailor the results specifically, then by all means use Google, that's what they're good at!

<comment><USER12> _> how exactly do you expect DDG to know what you want from this_

The obvious heuristic is "does this search term frequently appear on local result sites?" For example, "brick suppliers" is a search term associated with websites that have a business address, whereas "python string length" mostly isn't. The search engine already has to scan your search to know what to send you from the index; it doesn't require anything more than that to be able to guess at the intent of specific searches.

</comment></comment></comment></comment></comment><comment><USER13> I dislike Google as a company but their search results are orders of magnitude better than the competition. DDG or Bing are, to me, very bad (aren't they almost the same anyway?)

It's unclear why that is, and why search is so hard, but obviously it is.

<comment><USER14> Do you have an example or generally what kind of searches you are referring to? My experience with ddg has been "good enough" at least. The few times I don't find anything usable on ddg I ususally end up not finding anything on Google either

<comment><USER15> Not the poster, but here is an example. Elsewhere in this thread I posted a fact that I knew, but wanted to double check. It was about the LexisNexis news database containing 35k sources.

Here are the queries for "lexis news 35000" on Google and DDG:

[https://www.google.com/search?q=lexis%20news%2035000](https://www.google.com/search?q=lexis%20news%2035000)

[https://duckduckgo.com/?q=lexis+news+35000&ia=web](https://duckduckgo.com/?q=lexis+news+35000&ia=web)

The information I was looking for is in the first Google results. I didn't see it on the first page of DDG at all.

This was just my last search as an example without even trying too hard to come up with one.

I like DDG, but Bing (where the majority of DDG's organic results come from) is not as good as Google for organic web results IMO.

<comment><USER16> You are mistaken in assessing the situation, this has nothing to do with results, but with query composition.

[https://duckduckgo.com/?q=lexisnexis+35000+news+sources](https://duckduckgo.com/?q=lexisnexis+35000+news+sources)

You have trained Google to recognise Lexis as an alias for LexisNexis. DDG does not offer that, so one has to spell it out explicitly. You have to decide whether it's acceptable for you to change habits back to the common ground for Web searching.

The value provided by magic intent recognition (as opposed to traditional searching for keywords) differs from person to person. On the whole, it's likely for the better and that's why that feature was implemented at Google, but I have also read countless complaints about it on HN because situationally it pollutes results and can't be turned off.

<comment><USER15> _You have trained Google to recognise Lexis as an alias for LexisNexis._

I disagree completely. Google knows Lexis is an alias for LexisNexis, but I did not train it from an individual level filter bubble perspective.

Look at the below results. They are sourced from Google but proxied so that Google doesn't get any information on the end user. The firsts result should be one that provides the relevant article I was initially looking for. [0]

[https://www.runnaroo.com/search?term=lexis+news+35000](https://www.runnaroo.com/search?term=lexis+news+35000)

Just one final point. Half the results on the DDG page for initial "Lexis" search I ran are for the car company "Lexus". So let's not pretend that DDG is respecting user search queries.

[0] [http://www.lexisnexis.com.sg/en- sg/products/nexis.page](http://www.lexisnexis.com.sg/en- sg/products/nexis.page)

</comment></comment></comment></comment></comment><comment><USER17> DuckDuckGo is finally good enough to use as a daily driver.

About 2 years ago I tried it and I constantly found myself using !g to get decent results.

Now I only need to use !g some of the time for programming or technical searches. It works great for everything else.

I just wanted to add this because I was anti-DuckDuckGo for awhile until I recently retried it.

<comment><USER18> Yep. I switched to DDG last year and haven't looked back. Between Safari/Firefox, DDG, Fastmail, and Apple Photos I think I've moved myself almost entirely off of Google.

Recently set up a Pi-hole to shield me from the rest. What else am I missing? Oh, YouTube! Drat. I'm a sucker for hobby videos on there. I've seen a couple of alternative front ends but nothing that has really stuck so far.

<comment><USER19> How are you liking Apple photos, considering the switch myself

<comment><USER18> I'm happy with it. I'm not sure why I resisted turning on iCloud Photos. It makes cross device management a non-issue. Now I just need to find the time to cull my 20k something photos...

The major sticking point remaining is a shared library. I have my photos. My wife has hers. How do we manage photos of our children without duplicating?

</comment></comment></comment></comment><comment><USER20> While I am supportive of their business model, and search functionality, I find it off-putting the way they speak of privacy but then use maps provided by a GAFAM which are ultimately profit oriented and can _not_ be trusted to safeguard privacy regardless of their marketing. OSM would be a great choice here. I'll even dare to speculate that they were financially incentivized to make their choice in the face of a superior and actually privacy-oriented choice.

<comment><USER14> You can use different Map backends for the map functionality, OSM amongst them

<comment><USER21> Is there a way to do this for the maps search? I only see an option to change the directions provider.

</comment></comment></comment><comment><USER22> This is anecdotal, but I had a positive experience with the lack of privacy recently. I bought a baby stroller, after doing the typical research and finding a used model, I ended up buying a new one from some babyshop. All the pictures had shown the thing with the stroller "hat/head/sunscreen thing" (sorry I don't know the English word for kalche), anyway, it wasn't included and that was why the brand new stroller had been priced the same as the used ones.

But it's bought and it's nice, so whatever I hit the manufacturer website and find the correct product and google it, and get a flash sale from one of our most prominent baby stores. It was 350 Danish kr including delivery. Without closing the tap I check a few baby shops and a price checker website and see it's actually 600-700 Danish krs everywhere, including on from the company the flash sale on google is form. So I buy it.

Apparently I hit the right combination of search history, and store advertising/inventory at exactly the right time.

Being curious I called my local baby store to ask why they could flash sale me at half price, and after a bit back and forth they apparently do this thing where they'll catch you early with a cheap item and then when you come back they to buy it the next day it'll be priced higher, except by then you've made up your mind to buy it and will pay the extra and I was just lucky having already made up my mind when I got it because the other store has been cheeky.

Not really related to DDG, but it's the first time selling my privacy has paid off.

<comment><USER23> I feel like the Airline industry does this when they say, "only 4 more tickets left!" Or "2 tickets recently purchased!"

There's probably some terminology for this in game theory (well, general salesmanship too)

<comment><USER24> In the UX world they are usually called urgency or scarcity triggers [1][2]. They also play in to the social default bias [3] people tend to have, where when alone you tend to do things that you know others have done.

1: [https://www.researchgate.net/publication/259906084_Scarcity_...](https://www.researchgate.net/publication/259906084_Scarcity_messages_A_consumer_competition_perspective) 2 (PDF): [https://www.uvic.ca/socialsciences/economics/assets/docs/sem...](https://www.uvic.ca/socialsciences/economics/assets/docs/seminars/Online%20Scarcity%20Informational%20Value1015.pdf) 3: [https://www.coglode.com/gem/social-default- bias](https://www.coglode.com/gem/social-default-bias)

</comment></comment></comment><comment><USER25> While we are at it - what is the business model for freeware web browsers such as Vivaldi, Firefox, etc... I know they have some deals with MS, Google and friends but how such deals are done? Is it from some personal contacts or do you have to apply officially? What are the particular numbers - is it something you have to negotiate? What is the range in this case? etc...

<comment><USER26> Mozilla makes a referral fee from search engine searches made with the browser. Their revenue is in the hundreds of millions.

[https://www.cnet.com/news/google-firefox-search-deal- gives-m...](https://www.cnet.com/news/google-firefox-search-deal-gives- mozilla-more-money-to-push-privacy/)

<comment><USER27> So basically the advertising companies own the whole stack. Explains a lot.

</comment></comment></comment><comment><USER28> FAQ on their own site: [https://help.duckduckgo.com/company/advertising-and- affiliat...](https://help.duckduckgo.com/company/advertising-and-affiliates/)

<comment><USER29> SpreadPrivacy is an official DuckDuckGo blog too.

</comment></comment><comment><USER30> > What Is the Business Model for DuckDuckGo?

The difference between the revenue from Bing Ads they show and the money they have to pay Bing for their search API.

<comment><USER31> I'm surprised they didn't mention it at all on the page. It's definitely mentioned on other pages about how to get your ads to show up on DDG.

<comment><USER30> I'm not. Being a re-seller of search and ads is not an impressive narrative for a company that sells the illusion of a search alternative. DDG adds value in the privacy realm, and I'd rather have it exist than not, but its continued existence depends on the continued will of a search api provider to sell and in the western world there's basically only 2 in that game: Google and Bing.

</comment></comment></comment><comment><USER32> So first off personal bias, whize.co is the search engine I and my co-founder have been working on, it was nice to see basically our intended business model entirely validated by this article. Aside from the work we've done to validate it of course.

We entirely agree with their stated aims here but believe we can produce more quality search results than many current engines, hopefully it works out.

<comment><USER33> > This alpha is limited to GitHub results.

ok, so:

[https://alpha.whize.co/search?q=KubeConfigLoader&page=1&n=10](https://alpha.whize.co/search?q=KubeConfigLoader&page=1&n=10)

cheerfully says: "Showing 0 - 0 out of 0 for: KubeConfigLoader" although in your defense, it was blazingly fast to return that result. The shape of your XHR response seems to indicate you have an ES instance with only one shard, so maybe the site is still running in demo mode or something?

contrast that with: [https://duckduckgo.com/?q=site%3Agithub.com+KubeConfigLoader](https://duckduckgo.com/?q=site%3Agithub.com+KubeConfigLoader)

As a meta observation, in the bottom corner of every DDG result that I've ever received is a "Send Feedback" button, allowing me to advise them of bad search results in a very low friction way, but I had to come here to report my experience with your site

<comment><USER32> Yup still Demo mode the index is more than a few months old at this point. We're close to launching a beta with a much larger unrestricted index and a re-crawl strategy that keeps things fresh.

We've also changed our ranking methodology. The alpha / demo there was focused on novelty so if you search say "machine learning" you'd get repos that were recently active, starred and (hopefully) relevant to the topic, worth noting we also hard down-ranked repos that had been around for a while to let the smaller ones show through.

Through user conversation we moved away from focusing on novelty and moved onto "information staleness" the idea that elements in search results do not decay in relevance as quickly as they should based on real world events that happen around them. Happy to talk more about it if you want.

</comment></comment></comment><comment><USER34> It's a bit strange but for me Google's biggest pull is live scores.

I follow cricket, tennis and football. And all the dedicated websites are heavy and slow (cricinfo, espn etc). Google scores are quick to load and update even before the rest.

<comment><USER35> Posted the same elsewhere. I've been able to change my own behavior to look for sports reporting elsewhere. Agreed Google has done an excellent job of it.

</comment></comment><comment><USER36> I am going to build my own search engine but with a monthly subscription fee. No ads, no tracking and I don't keep logs of your activity. So across between a VPN and a Search engine!

<comment><USER37> I admire the initiative but there is no way people will pay for a search engine.

<comment><USER38> Quite the generalisation..

To provide a counter generalisation, I am sure a certain percentage would pay for a niche search engine if it provided specific value.

I mean, LexisNexis news search exists, and is far from free.

<comment><USER39> Because Lexis has a massive database of news stories and the rights to share it.

<comment><USER40> I would wager that Google's database of new stories is bigger. LexisNexis survives partly through institutional inertia, but also I expect because they've tailored their search product to their specific niche in a way that is much more useful than Google's general-purpose algorithms.

<comment><USER15> _I would wager that Google 's database of new stories is bigger._

You are correct. Lexis only has 35k news sources. Their one strength is that their news archive goes back to the late 70s.

The real difference is that Lexis's core business is B2B. Enterprise pricing defies the laws of reason for what people will pay for software vs B2C.

<comment><USER40> Google's news articles goes back way further than the 70's. They've been scanning newspapers and integrating digitization archives. This is why you can you can look at word usage trends centuries back. But it's not very well integrated into search.

</comment></comment></comment></comment></comment></comment></comment><comment><USER41> I saw a DDG billboard in my city the other day. I was really surprised they did any type of advertising like that!

</comment><comment><USER42> DDG is surprisingly good, but still not good enough. I have my desktop browser using it, but still haven't on mobile.

When searching for technical queries Google tends to give much more useful answers - useful is the key word here. To give an example from yesterday, the query 'elixir zlib'.

In DDG I get links to C files in Linux kernels on a server called elixir. I get a useful answer 5 results down (which is fine, it's there). In Google, the first result I get is the erlang zlib documentation (which is what you actually use in elixir).

Things like error messages, queries involving the name of a programming language etc. are often just not as good in DDG.

<comment><USER43> > Things like error messages, queries involving the name of a programming > language etc. are often just not as good in DDG.

I don't have any problem with those types of searches on DDG. Even obscure error messages turn up most of the time, and when they don't turn up in DDG they don't turn up in Google either.

Even when I do searches for a relatively less popular language like Haxe, I don't have trouble finding what I'm looking for. I only remember a handful of times over the 3+ years I've been using DDG that Google returned an ancient IRC log referencing my issue while DDG didn't return anything, but just the same I've had Google return irrelevant results while DDG gave me exactly what I was looking for.

Since my anecdotes offsets your anecdotes, the only conclusion is that anecdotes are useless to compare these things :)

</comment></comment><comment><USER44> From the article: "Alarmingly, Google now deploys hidden trackers on 76% of websites across the web to monitor your behavior"

Its not Google that deploys these, it's webmasters who want free Google analytics. Big difference.

</comment><comment><USER9> I think it would be cool if one started an online advertising company in 2020 that employs about 50% graphic designers and 50% other and that specialises on providing online JPEG or similar images for websites, and where each add is completely human generated / human curated.

Also, the sizes should be completely fixed like you can choose between predetermined pixel ratios. Like maybe 162x100 or 1600x30 or whatever.

With people _angsty_ with social media, I think it could be quite refreshing to have a trip back in time.

</comment><comment><USER45> I find it very interesting that this article isn't just all "We're more holier [SIC] than google", but instead also argues that google could just as well adopt this busyness model. It doesn't defend google, but it doesn't paint it as this intrinsically evil IT-Monster that must be defeated, as many privacy advocates tend to do.

</comment><comment><USER46> If anything, I feel DDG should get a more serious name, and one that doesn't sound funny. Seriously, whenever I hear DuckDuckGo, I feel it sounds like some blog or some meme sharing site. This only hurts its brand value and make it seem trivial.

</comment><comment><USER47> One huge thing stopping me switching over is not being able to see Google reviews for restaraunts, doctors, etc. Yelp is, in my opinion, much less trustworthy and it would be nice to be able to toggle this the same way Google maps can be a preference.

</comment><comment><USER48> Some commentors are saying Google is worse. As someone who uses Google, what's been worse about it? With regards to ads, I don't see them as I use uBlock Origin on desktop and mobile (as well as other ad blockers and no-script extensions).

<comment><USER49> What's worse is that these people hate Google. Many do because they failed the interviews there and openly state this as the reason for their criticism.

</comment></comment><comment><USER50> I think Google only keeps DuckDuckGo around so they can claim they have "competition."

<comment><USER51> Why would it be up to Google to decide whether or not DuckDuckGo gets to "be around"?

<comment> maze-le I am also confused about this wording but I assume it refers to the duckduckgo-app on android devices. It would be blatantly anti-competitive if google would refuse the ddg app in their app store, solely because they compete with them in the search-engine market -- there are legal mechanisms against this behavior.

<comment><USER51> Agreed. And there are laws against this sort of thing, and the regulators are slowly but surely catching up to the fact that they will have to start actually enforcing them.

</comment></comment></comment></comment><comment><USER52> Interestingly, DuckDuckGo allows you to turn off advertisements in their settings.

[https://duckduckgo.com/settings#general](https://duckduckgo.com/settings#general)

</comment><comment><USER53> I use DDG and hardly ever use Google. Apart from privacy, I like one feature which is searching news by date. Anybody had experience with their API?

</comment><comment><USER54> This is some good news. I was worried that they are not making enough money and might shutdown. I will continue using it.

</comment><comment><USER55> It would be nice if DuckDuckGo supported IPv6, it does not appear to use an AAAA record in DNS.

</comment><comment><USER56> I've been seeing DDG billboards all over, which seems weird.

<comment><USER57> I just saw my first one yesterday and made me instantly question how they paid for that and what their business model was.

</comment></comment><comment><USER58> Tldr:

DuckDuckGo is profitable

\- mostly by keyword-based search ads

\- also by affiliate links to Amazon e.a.

</comment><comment><USER59> I've always had a bad vibe about duckduckgo

<comment><USER60> Less than ideal case-scenario, users are spreading their data between Google and Duck Duck, Bing etc. Not empowering a single agent. In the worst case Duck sell our data to Google. I find that unlikely.</comment></comment>


Want to see all data Windows 10 sends Microsoft? - <AUTHOR> https://arstechnica.com/gadgets/2018/01/want-to-see-all-data-windows-10-sends-microsoft-theres-an-app-for-that/ <comment><AUTHOR> I remember using bitvisor (a very thin hypervisor like Xen but for intercepting sys-calls or hyper-calls) to see if Windows XP and Windows 7 send anything without being caught by the OS level firewall. It was actually but I never investigate further. So with a grain of salt, I would never trust what Microsoft provides a way to increase the trust of people who are worried. Also in this documentation, it doesn't mention what is the data actually but it only says some keys and ids.

I would like to see how the OS I have paid for (not everyone is just upgrading) can be stopped of monitorings. Granted I am coming from Linux and Mac but it took me a lot of time to find the checkboxes and there is still no way to know if I have unchecked them all and if they are adding more of these in the future updates. Which may be this tool can be helpful for me knowing oops I missed one of those apparently.

</comment><comment><USER2> Yeah, I'd rather just block all of that stuff and monitor it on my own, but thanks for trying Microsoft. Or maybe I should thank the EU for the GDPR and the DPA's for forcing you to do this.

<comment><USER3> Can you provide some proof for these requests ? I am not able to find the legal requests on internet..

</comment></comment><comment><USER4> It would be an interesting test to see how much data gets sent from your average heavy computer user. I use a ton of software, and I always opt out of the request to send telemetry data to "help improve the software".

I was feeling bad the other day for not allowing a nice open source program to do so, but realized that my upstream line might be completely saturated with constant telemetry data if I didn't always opt out.

</comment><comment><USER5> How about Microsoft just not send any data?

<comment><USER6> How much do Apple and Google send, I wonder.</comment></comment>


Anonymous programmers can be identified by analyzing coding style - <AUTHOR> https://freedom-to-tinker.com/blog/aylin/anonymous-programmers-can-be-identified-by-analyzing-coding-style/ <comment><USER2> Interesting to see this formalized. When I was in grad school and graded undergrad homework/exams, I could most definitely recognize the students by their coding styles after just a few assignments. Every student ends up developing their own habits, and they're quite easy to spot in something as repetitive as code.

I remember teaching a Matlab class for engineers and scientists that was about 50/50 male/female, and the women tended to have much neater code. Code written by males often had comments all jumbled up, inconsistent number of spaces between braces/operators/etc, incoherent variable names, worse names for functions, and so on.

<comment><USER3> It's funny to think that if you reversed the genders the comment would at the bottom of the page rather than the top.

<comment> vertex-four On HN? Definitely not.

</comment></comment></comment><comment><USER4> My Code Jam solutions always _shared_ a lot of code -- all the boilerplate for reading inputs, parsing integers, iterating over test cases, and writing out results.

Because of that, it seems like Code Jam is an artificially easy test case for this sort of identification -- I'm pretty sure a _human_ could look at my solutions and conclude they were obviously all written by the same person.

</comment><comment><USER5> Not adhering to style guides is now a privacy issue.

<comment><USER6> Is it odd that I've been thinking about this one for a while now. I don't really talk to blackhat folks on the regular anymore, but given how important OpSec is, it's really sad how easily identifiable most of their code ends up being. Malware authors brag like crazy ("oh, let me just go ahead and encrypt this payload with my handle as the key...") and I wouldn't be surprised if you found some of the same from the state-sponsored folks.

I'd really love to see someone like grugq weigh in with some thoughts here. The idea of having tools to parse and rewrite my code to be as generic as possible came to me years ago.

<comment><USER7> > The idea of having tools to parse and rewrite my code to be as generic as > possible came to me years ago.

You basically want an obfuscator that replaces all the names of things with generics, and then randomly permutes blocks of code without changing the code paths possible in the final binary. (Perhaps some optimized-for-performance version of this, but that might identify the tool you use.)

It sounds relatively easy to write if you stick to certain coding guidelines (like using techniques amenable to static analysis).

However, this still won't work in some cases, because you'd need more advanced tools to handle profiling of what sized functions and such you ended up writing.

It would be interesting to try and write a tool which defeated any analysis of author patterns in the code, but would require understanding the program across the boundary of function calls, which is a difficult problem. (You probably couldn't write Turing complete code, for example.)

</comment></comment></comment><comment><USER8> Reminds me of how telegraph receivers used to be able to identify transmitters of the telegraph by their "fist" (cadence or rhythm with which they signaled).

Here's a Schneier post about a Concordia University study about identifying e-mail authors. [https://www.schneier.com/blog/archives/2011/08/identifying_p...](https://www.schneier.com/blog/archives/2011/08/identifying_peo_2.html)

</comment><comment><USER9> But can they tell us if TJ Holowaychuk is really only one person?

<comment><USER10> Could someone provide a comprehensive description of what on earth is TJ Holowaychuk? I did Google a bit, and I'm still confused. Is it the new __why?

<comment><USER11> No, he's just ridiculously prolific.

</comment></comment></comment><comment><USER12> It's interesting that they only analyze the abstract syntax tree and ignore formatting. I would suspect that brace placement, tabs vs spaces, etc. would provide a useful fingerprint as well.

<comment><USER13> They used both:

> We used a combination of lexical features (e.g., variable name choices), > layout features (e.g., spacing), and syntactic features (i.e., grammatical > structure of source code)

The AST stuff is super interesting. The other signals are somewhat superficial. But comparing ASTs? That is deep.

<comment><USER14> One would think that, but with AST's generated from 'normal' text (not code), they're actually quite noisy, and lexical and syntactic features have been more useful[1]. In traditional authorship attribution, AST's are a about decade-old technology. But then, this is code, so for all I know, very different.

PS: 1] shows that using AST's does does not get you THAT much of entropy gain compared to other features.

1: [http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6234420](http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6234420)

</comment></comment></comment><comment><USER15> They have done a presentation at the 31C3 a few days ago on this presenting their findings in more details & Q/E ([http://media.ccc.de/browse/congress/2014/31c3_-_6173_-_en_-_...](http://media.ccc.de/browse/congress/2014/31c3_-_6173_-_en_- _saal_g_-_201412291715_-_source_code_and_cross-domain_authorship_attribution_- _aylin_-_greenie_-_rebekah_overdorf.html#video) ).

What I understood from that - it worked quite well with code bases like from the Google Code Jam (large LoC, no style guides etc), but not that well with smaller amounts of code and I'm looking forward to some additional results e.g. with a codebase from a corporate development environment.

</comment><comment><USER16> I think that kind of analysis wouldn't be possible with Go ([http://golang.org/](http://golang.org/)), since it is very strict/limited and uniform.

<comment><USER17> I think that one of the most important parts of this analysis is the naming of variables, so I guess it still be possible (though probably more difficult)

</comment></comment><comment><USER18> This doesn't surprise me. When I was in college we had a daily paper I worked at the photo dept. We has a box of "feature Photos". they were kind of filler (campus life, people playing hacky sack, feeding ducks , setting up for events etc..) I figured out one day could look at the photos and tell who took them by the style (the photographer name is on the back).

At one job I had we called uncommented, poorly formatted code "curtis code" for some reason.....

</comment><comment><USER19> Of these 3, at least 2 would depend on the language in use:

"We used a combination of lexical features (e.g., variable name choices), layout features (e.g., spacing), and syntactic features (i.e., grammatical structure of source code)"

In particular, "layout features" is a huge issue in some languages, and not at all in others. For instance, a language like Javascript, or PHP, give great flexibility about layout, so in those languages I can see each developer having a unique style (and I have been involved in style debates regarding those languages), however, a language like Python has a fairly fixed layout, since the whitespace is significant. And also, in Clojure, I think most programmers use Emacs and accept the Emacs clojure-mode indenting as the default.

Variable name choices is another where some environments encourage similarity, and others allow for unpredictability and unique styles. Within the Ruby On Rails framework, for instance, there are norms about the creation of variable names.

I would guess that syntactic features is perhaps the one characteristic that shows a great deal of uniqueness in every language. I am often surprised at the choices my fellow co-workers make, when it comes to how to solve a problem.

<comment><USER20> Python does not have anywhere near a fixed layout.

Off the top of my head:

* When / if single-line if / while / etc statements are used * How many blank lines are used between functions * How often blank lines are used in functions * How much indentation is used for initializing lists / etc. * If multiline strings are used.

Etc.

Some of these are covered by PEPs, yes, but enough people don't follow PEPs religiously that even those offer some information.

</comment></comment><comment><USER21> _Could_ be identified in some cases of amateur code, like PHP or Javascript or Clojure.

Good practice is to follow a very explicit coding style which makes code written by different developers indistinguishable - the more the better.

Go ahead, identify which developer wrote which part of Linux kernel or, god forbid, jdk/src/ _

</comment><comment><USER22> Something that would be interesting is to follow code styles across people who've pair-programmed. Kinda like the apprenticeship model, I wonder if you could detect specific styles that get adopted and evolve over time.

</comment><comment><USER23> This is fascinating, my mind is immediately drawn to simple obfuscation programs that would turn tabs into spaces and change the formatting and so on, while still leaving it syntactically correct. Not obfuscation such as to hide the purpose of the code, just the identity of the author.

Does anyone know of any such projects, or what they might be described as? None of the queries I've tried produce the intended results.

You could even take it one step further, if you can identify the author of source code, can you not then forge that signature to make it look like they wrote something they didn't?

<comment><USER24> _Does anyone know of any such projects, or what they might be described as? None of the queries I 've tried produce the intended results._

I guess it's because you're looking for "obfuscators" while such programs are usually known as "automatic code formatters"... and any decent IDE is going to have the functionality to do this.

For something standalone, look at [http://en.wikipedia.org/wiki/Indent_(Unix)](http://en.wikipedia.org/wiki/Indent_\(Unix\))

<comment><USER25> Or astyle for more versatile standalone formatter (and it's frequently the one used by IDEs)...

</comment></comment></comment><comment><USER26> I have a question, now I've never decompiled anything but I was under the impression it would come out in machine code, so you wouldn't get programmers notes, tabs etc. Can someone explain how their doing this? I knowI should know this Haha, be gentle :-p

<comment><USER27> You wouldn't get any if that information unless you ran -g with gcc. However, the article uses the source code and doesn't decompile anything.

<comment><USER26> Ahh thanks guys I must have missed that part.

</comment></comment></comment><comment> ryan-c I'm pretty sure something similar could be done with shell history logs.

</comment><comment><USER28> This stylometry analysis is 95% of a stylometry obfuscator/homogenizer.

</comment><comment><USER29> The same way authors of text posts online can be identified.

</comment><comment><USER30> gofmt ftw ;)

<comment><USER31> Formatting along those lines are just one vector that's used. This gives a lot of weight to the underlying AST of the code, so in order obfuscate, you'd have to have a program that scrambles variable names, placement and content of control blocks, etc. Basically, you'd need less an autoformatter and more an obfuscator, probably coupled with a deobfuscator to make the code as generic as possible.

</comment></comment><comment><USER29> Can it help to find the real author of bitcoin?

<comment><USER32> Hard to say. This is analogous to drug testing or terrorist hunting: even if you have a highly accurate test, you're going to want to apply it to thousands of programmers, and suddenly, when you do the Bayesian calculation, your high accuracy turns out to still be a low probability of having correctly identified the true author.

And then you have to justify your closed-world assumption: how do you know Satoshi (under his real name) was _even in your dataset_? Maybe after Bitcoin he went back to closed-source work or commercial projects, and none of his source code other than Bitcoin appears in your dataset. Then the guy your analysis picked out isn't 'Satoshi' so much as 'the guy who looks the most like Satoshi (but actually isn't)'.

<comment><USER29> We can find a number suspects, and then analyze other facts about them.

</comment></comment></comment><comment><USER33> "Prose authorship attribution that utilizes parse trees have been able to identify an anonymous text from 100,000 candidate authors 20% of the time."

Color me unimpressed</comment>


Same Stats, Different Graphs: Datasets with Varied Appearance and Identical Stats - <AUTHOR> https://www.autodeskresearch.com/publications/samestats <comment><USER2> I don't think the title does a great job of identifying how cool this is. Having the animations especially is great!

<comment><USER3> Thanks! I'm the author of the paper, glad you like it, and especially glad that you think the animations are cool :-)

<comment><USER4> Congrats on the paper, very cool work. I bet the "Datasaurus dozen" is going to become the go to example for the dangers of relying on summary statistics, it is much more memorable than Anscombe's quartet.

<comment><USER3> Thanks! And here's hoping :-)

</comment></comment></comment></comment><comment><USER5> > Boxplots are commonly used to show the distribution of a dataset, and are > better than simply showing the mean or median value. However, here we can > see as the distribution of points changes, the box-plot remains the same.

Violin plots [1][2] are a great spin on boxplots that help show the distribution.

[1] [https://en.wikipedia.org/wiki/Violin_plot](https://en.wikipedia.org/wiki/Violin_plot)

[2] [http://seaborn.pydata.org/generated/seaborn.violinplot.html](http://seaborn.pydata.org/generated/seaborn.violinplot.html)

<comment><USER6> But a violin plot still wouldn't distinguish between many of the plots in this set. All of the plots that don't have multiple vertical stacks -- so, all the horizontal lines, diagonal lines, circles, and scatter -- will have the same or almost the same violin plot.

You could rotate the violin plot to show how the density changes on the other axis, for those graphs where that would be useful, but that requires looking at the data visually and making decisions, which is the whole point of the article.

<comment><USER5> Agreed. But my response was specifically in response to the "More Examples" section that shows three examples for which the boxplots are ambiguous but violin plots would not be.

In general, scatter plots can be very valuable but difficult to compare among trials. Violin plots help mitigate that, though you're correct that it comes at a cost.

</comment></comment></comment><comment><USER7> I'm wondering how the general public can be educated about this when so many people are unaware even of the difference between the median and the mean.

In the mass media especially, the mean is often bandied about as the only statistic, and treated as if it was definitive.

<comment><USER8> I think the public is largely aware of the difference since housing prices are often reported using the median. Intuitively, everyone knows that median housing price is the price of a 'regular' (most frequently sold) house in a given area.

<comment><USER9> The most regularly encountered house price at sale would be the mode.

Median is the value at which 50% of houses trade higher as well as lower.

</comment></comment></comment><comment><USER10> You see this a lot in the context of building a regression and often times the assumptions are violated:

* Linear relationship between predictor and response and no multicollinearity * No auto-correlation (statistical independence of the errors) * Homoscedasticity (constant variance) of the errors * Normality of the residual(error) distribution.

As the paper suggests, plotting the data visually will help you avoid these assumptions, but also just making sure you don't violate the assumptions w/ statistical tests would work too. For example, uou can look at your residuals (loss) as an indicator of good fit. If your residuals do not follow a normal distribution, this is typically a warning sign that your R2 score is dubious.

There are a few statistical tests for Residual Normality, particularly, the Jaque-Bara test is common and available in scipy.

So, I would argue, you don't even need to visualize the data. I describe this more here: [http://www.eggie5.com/104-linear-regression- assumptions](http://www.eggie5.com/104-linear-regression-assumptions)

<comment><USER11> Sure, using hypothesis tests could pick out some of the structured examples in the Datasaurus, but in practice, things are often more subtle. Goodness of Fit tests to check for normality, in particular, are a little bit thorny, lacking power in small sample sizes, and rejecting normality for slight departures in higher sample sizes. My experience has been with assumption checking that by the time a hypothesis test has sufficient evidence to reject an assumption, you'd usually be able to see it visually.

Until you get into high dimensions, it probably doesn't hurt too much to visualize the data. Additionally, it can be helpful to understand what signal has been left in the residuals (ex: you fit a linear model, but failed to include a quadratic term), which is something hypothesis tests aren't as good at telling you.

<comment><USER10> Yes, "lacking power in small samples sizes"

</comment></comment></comment><comment><USER12> You can see the statistics for each data set here (in GeoGebra) [https://www.geogebra.org/m/K8HCdE6X](https://www.geogebra.org/m/K8HCdE6X)

</comment><comment><USER13> Awesome project, I really like it, so this is my attempt to achieve similar result with different approach.

I feel it might be an useful example for illustrating the intuition of ZCA and Wasserstein metric.

[http://nbviewer.jupyter.org/github/tjwei/Animation-with- Iden...](http://nbviewer.jupyter.org/github/tjwei/Animation-with-Identical- Statistics/blob/master/Animation%20with%20Identical%20Statistics.ipynb)

</comment><comment><USER14> Good use case for n-dimensional probability density functions. Query a region by a hypercube.

</comment><comment><USER15> Awesome article. I just read with my morning coffee, and I have a big grin.

<comment><USER3> Thanks! Glad you liked it :-)</comment></comment>