The Untold Story of Chegg PT 1 - <AUTHOR>

My name is Ousama Haffar and I was one of the early guys at Chegg.
While Chegg isn't a Zynga or a Facebook story that attracts the oohs and ahhs of Silicon Valley, I would love to share a bit with you about what I learned about entrepreneurship and being scrappy during my time at Chegg as founding member of the team from 2006-2010 and how we went from $0 to over $100M in revenue.
Chegg originally start its life as a college classified site that attracted 60,000 students across the country to buy and sell your everyday college type items; t-shirts, mini fridges, old beat up Pontiacs, etc.  Our team was about 5-7 people, mostly engineers and I focused mostly on product, marketing and whatever odd jobs that needed a hand.
In our early days we attracted a couple hundred thousand visitors to our website each month, mostly buy building a campus marketing team and doing your traditional marketing. We would give away pizzas, flyer in dorms work with student organizations to create swap meets at the end of the semester and even created a textbook exchange.
Well, back in 2006 the name of the game in Sillicon Valley was user generated content and eyeballs. We dreamed of building this vibrant and massive classifieds business that could be monetized by advertising in each vertical in the already $200BN college market. In reality, we hit the same problem most marketplaces hit, the good ole' 80-20 rule.  Users seemed to love browsing the site but never felt like posting items for sale. And as a result we hit a growth ceiling.  Well, back in 06 and 07 with Facebook's popularity storming off and sites like Youtube changing the world, most web startups felt most strongly about generating users, and well when your user growth hits a wall...you're in trouble.  So we did what any startup with a million bucks in the bank looking for quick fix to growth problems would do, Marketing Promotions.
So, me being the marketing/product guy thought hmmm...what would college students want that would make them tell all their friends and magically grow our user numbers overnight? In 2007 that answer was the hottest item on the market, the Nintendo Wii. So with a couple thousand bucks in hand and a skill for finding sellers of the hottest gift on the planet we went to work and secured a supply of 6 Wiis.  Here comes the kicker, we planed a promotion to sell them for the insanely low price of $99 on our wonderful classifieds website and promote it using Deals of America, Fat Wallet and sending an email to our user base.
What happened was predictable, but nevertheless exciting. A fresh line of oxygen and the beginning of my misery.  Within an hour after launching our promotion the website came to a standstill, our server couldn't handle the traffic and our best case scenario turned into a nightmare.  Meanwhile, some college hackers thought it would be fun to hit our site with a D.O.S and as soon as we were up and attracting hundreds of thousands of users  we were down and scrambling to get it back up and running.  The pure exhilaration of seeing an idea become an instant hit to turning away tens of thousands of users was just a taste of the the roller coaster of emotions I would experience over the next 3 years at Chegg.
We finally got the site back up and running and within seconds the Wiis sold out. We thought to ourselves wow, our "Hatch of the day" as we called the promotion was a mega hit and it's time for us to start repeating it day in and day out.  With a $50,000 budget I went to Frys, scowered amazon and found the most popular college items of the day and loaded a back room with marketing inventory for our daily deal sales.  Within the week the website seemed like a vibrantly growing community, the word was quickly spreading and our story to investors seemed bright and optimistic, the day was saved thanks to the Hatch of the Day, or was it...
Part 1 of 6 <comment><USER2> My friends in college swear by Chegg. You've done a tremendous job building loyal customers. Looking forward to hearing the rest of this story. Could you drop me an email (I didn't see any contact info in your profile) firstname.lastname@example.com? I'd love to bounce something off of you.

</comment><comment><USER3> I used to fill out Chegg mturk requests for reviews while talking on the phone. They were way overpriced and nobody was paying attention to quality.

</comment><comment><USER4> I'm really keen to read parts 2,3,4,5, and 6.

I know someone working for Chegg and they love it there.

</comment><comment><USER5> Keep it coming. I love things like this.</comment>


Solar panel power is rocking it in the U.S. in 2012 - <AUTHOR> http://gigaom.com/cleantech/solar-panel-power-is-rocking-it-in-the-u-s-in-2012/ <comment><USER2> I am pretty excited about this field. I have been working with some folks who are developing controllers for off grid systems. My personal vision is to develop small systems that can be added modularly. First a few panels to run a DC pool pump, then a few more to run a DC Air conditioner. As always, the hardest part is to consistently put in the time in between all the other things in life.</comment>


I created a free SaaS/Admin template for React - <AUTHOR> https://github.com/dunky11/react-saas-template <comment><USER2> I think it's better to list this under "Show HN"</comment>


From Dev Bootcamp to Engineer at Hipmunk: An Interview with Phil Aquilina - <AUTHOR> http://devbootcamp.com/2012/12/18/From-Dev-Bootcamp-to-Engineer-at-Hipmunk-An-Interview-with-Phil-Aquilina/

<comment><USER2> Ihanks for sharing your story. It was very inspiring to know you can learn at such a high rate and get a job you enjoy in roughly a year of programming.</comment>


Kubernetes LEMP Stack - <AUTHOR> https://github.com/chepurko/k8s-lemp <comment><AUTHOR> Just released a stable version (v1.0) with no TLS. Working on getting kube- lego to issue certificates automatically and do HTTPS exclusively. Would be awesome if I could get more eyes and brains involved.</comment>


Problems with CSVs - <AUTHOR> https://donatstudios.com/Falsehoods-Programmers-Believe-About-CSVs <comment><USER2> CSV is less a file format and more a hypothetical construct like "peace on Earth".

<comment><USER3> That's not a bad thought. Maybe RFC4180 shouldn't have defined csv, but another similar alternative.

<comment><USER4> CSV is inferior to TSV, there's really very little reason to keep using CSV other than taking advantage of its ubiquity.

<comment><USER5> It's a shame we don't use the ASCII delimiter characters. It would have solved a lot of the quoting issues.

[https://en.m.wikipedia.org/wiki/C0_and_C1_control_codes#Fiel...](https://en.m.wikipedia.org/wiki/C0_and_C1_control_codes#Field_separators)

<comment><USER4> Agreed, that would be the best delimiter choice for plain text data files.

</comment></comment></comment></comment></comment><comment><USER6> ASCII defines a unit separator, and a record separator. I wish we would just bite the bullet and start using this 50 year old solution.

Modern editors/spreadsheets could insert the characters without troubling the user too much, and show visible glyphs when that make sense.

[https://news.ycombinator.com/item?id=7474600](https://news.ycombinator.com/item?id=7474600)

</comment><comment><USER7> Falsehood: You can safely name your first column "ID".

[https://support.microsoft.com/en- us/kb/323626](https://support.microsoft.com/en-us/kb/323626)

[https://news.ycombinator.com/item?id=12041210](https://news.ycombinator.com/item?id=12041210)

<comment><USER5> Hah, I've spent years munging broken csvs and this one hit me on our first public data release.

I still maintain my release was fine and I blame excel ;)

</comment></comment><comment><USER8> We recently received a CSV from a bank where they didn't escape properly, so one of the entries was like this:

        "","C/O COMPANYNAME"SUBHEADLINE"","",(...)

So yeah, they probably wrote this themselves, and didn't expect a valid company name to contain these characters.

We ended up writing a custom parser around their horrible format, because their technical support and technical lead person we could reach couldn't understand our issues.

:(

<comment><USER9> What you describe is _exactly_ the problem with CSV: many less tech-savvy people decide to use it to provide some sort of very minimal API (even if it's periodical manual exports sent by mail; some people consider that automation!), don't account for the possible traps (like double quotes in the value of any field), then call it a day -- more like a lifetime.

Then there's inertia, there's a boss saying "we absolutely must work with these people", and... the rest is history for many of us.

NOTE: I've been writing custom CSV parsers at least 10 times during my career. That makes roughly 2 such parsers every 3 years. And that comes from a guy that's not very active in his career.

<comment><USER10> Not just with CSV. I remember a similar comment about broken XML.

It actually doesn't matter how compliant you are to the standard. Standards are certainly nice when both parties are willing to agree on things. But when one party is in position of strength and wants to minimize its costs or efforts, compliance to a standard won't help you.

<comment><USER9> That's the sad truth. Sometimes the thought of this makes me wanna go full Hitler on the internet -- if you can't expose a standards-compliant API, your service will be blocked by central tech authorities.

We can dream, right?

Still though, all giants eventually fall when they become phlegmatic. Sadly it's taking way too long.

</comment></comment></comment></comment><comment> nommm-nommm >All CSVs are readable with Excel

>Excel is a good tool for working with CSVs

>Excel is an OK tool for working with CSVs

Also See: Excel: Error when CSV file starts with "I" and "D" [https://support.microsoft.com/en- us/kb/323626](https://support.microsoft.com/en-us/kb/323626)

HN discussion: [https://news.ycombinator.com/item?id=12041210](https://news.ycombinator.com/item?id=12041210)

</comment><comment><USER11> > Excel is a good tool for working with CSVs > Excel is an OK tool for working > with CSVs > Excel can losslessly save CSVs it opens

Seriously. It's amazing how fast excel can corrupt a simple csv file.

<comment><USER12> I've had odd bugs show up in a CSV import process, where it was a numeric string (think account numbers) that had leading zeros. Turned out the imported CSV had been cleaned by moving records around in Excel and saving as CSV (which of course saw the account numbers as numerics and conveniently truncated the zeros)

<comment><USER13> Excel thinks anything vaguely scientific notation-ish is a float, which is a nightmare if you have, for example, four hexadecimal digits 2E15 which is 11797 in decimal but excel will helpfuly "fix" it to 2 million billion (depending on your locale of billion LOL)

That was a contrived example but I've seen Excel get very confused by foreign postal codes involving digits surrounding a letter "E" or apartment number columns for example "1E"

<comment><USER14> Excel enthusiastically and automatically types oh-so-many things. It'll take dates and assume they are in american MM/DD/YYYY in some fields and then treat dates in the same column like 13/01/1986 as strings.

That exact problem I had when working with a company who had a stocked item with the code "060E2". When people don't know scientific notation the result is bewildering!!!

</comment></comment></comment></comment><comment><USER15> This should be titled "What Vendors Should Account For When Publishing CSVs". If someone claims they're exporting a file in X format:

* X should be a known standard OR

* they should provide a client parser for X OR

* they should provide a grammar or list of rules for parsing X

This rant is focused in the wrong direction. It should go without saying that someone should deliver what they claim to. If CSV means different things to different people, that's fine too, but the burden should be on the exporter to define what it means to them.

I'll accept a retitling of "Bullshit Developers Shouldn't Have to Question About a Common File Format".

</comment><comment><USER16> It's literally impossible for CSV to be a useful format if you cannot make any of these assumptions. Every field with a distinct encoding? Really? The focus needs to be on making CSVs about which you can reasonably assume a lot of these things so that you can actually write software that processes them.

</comment><comment><USER17> Is there anyone out there who actually believes all CSVs are UTF-16? That's such a weird idea that I have no idea how somebody would arrive at it.

That's what always gets me about this genre of list — I often find myself scratching my head, trying to figure out if I've missed something or if they really listed that thing as a falsehood programmers believe.

<comment><USER13> Possibly its asymmetric political power at its finest but I am aware of at least one "business intelligence" application that outputs its CSV files consistently as UTF-16BE (not LE)

I'm not even sure what planet its from, much less the application, I just get stuck importing what they feel like sending me.

I don't know why everything that upstream sends me is in UTF-16BE format, its just what they do. Perhaps they hate me, or my employer. I donno. Perhaps its an intelligence test to cut down on support costs, you must be this smart or smarter to mess with our data or go away.

<comment><USER18> > Perhaps its an intelligence test to cut down on support costs, you must be > this smart or smarter to mess with our data or go away.

I like this design choice. I'll keep it in mind ;)

</comment></comment></comment><comment><AUTHOR> I question the title change from "Falsehoods Programmers Believe About CSVs"

It's not really a list of problems as much as incorrect things people think and simply reading that title alongside the list without the page title/description would make no sense at all to the reader.

I guess the HN gods know best though.

</comment><comment><USER19> One of the most common mistakes is delimiter in a field. E.g. comma as thousand separator...

<comment><USER20> Particularly problematic with this is that a human cannot necessarily determine with 100% confidence where the error is. If you have a CSV where every row has 5 entries, and you come upon this row:

> 100,1,200,300,10,100

You suspect that an in-field comma is causing a split where there shouldn't be, but who's to say which number got split up? 1200 into 1,200, or was it 200300 to 200,300, or 10100 to 10,100?

</comment></comment><comment><USER21> Having developed on a system that had to import and collate insurance claims data from multiple providers, I've come to dread CSVs for all of these reasons.

</comment><comment><USER22> One thing to remember if your company offers a customer-facing "export to CSV" button:

The format of that CSV is part of your public API and changes to it can cause pain for your customers. Don't be caught off-guard if they've built a workflow around "download this CSV, put it into this excel spreadsheet and run this VBscript on it" or "download this CSV into this iPython notebook and feed it to FOO".

</comment><comment><USER23> I remember at my last job, mentioning something about saving a Google Sheet in CSV format, and my boss immediately said "I'm morally opposed to CSV."

He then told me about TSV, and since then I've come to accept it as a superior format. With the kind of data I dealt with at my job, we never encountered tabs within a column, but commas within columns were very common, so TSV was perfect for us. A lot of what I was doing was ad-hoc processing of various sheets (nothing that was really production-grade), so a lot of time I'd just write code to read in a file and split each line on \t, and that was good enough. And maybe I'd throw out any lines that began with #, but that was about it for processing.

Actually, we _did_ have a TSV parser in our codebase, but my boss told me that it has problems and he never liked it and that I should just roll my own inline with my code.

</comment><comment><USER24> This has a lot of good stuff in it, but I think it's hurt by including stuff where you just have to make an assumption, E.g. I can interpret the following:

      "Some\tText"

in at least three different non-ridiculous ways, even assuming the encoding is known. It's worth keeping this in mind, but intertwining it with other _tractable_ assumptions feels like it hides some of the super-useful nuggets contained within. That said, I'll admit the dividing line is fairly indistinct, plus it's still a great list.

</comment><comment><USER25> Any known web (JS) workarounds for parsing CSVs with inconsistent file encodings? Right now I am at the conclusion that you must know the encoding type before parsing, though I've come across some decent libraries that can "guess" the encoding type:

[https://github.com/aadsm/jschardet](https://github.com/aadsm/jschardet)

[https://github.com/sonicdoe/detect-character- encoding](https://github.com/sonicdoe/detect-character-encoding)

<comment><USER26> Not sure what you mean by "inconsistent", but in general you have to know the character set before you read the file.

[https://www.joelonsoftware.com/2003/10/08/the-absolute- minim...](https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum- every-software-developer-absolutely-positively-must-know-about-unicode-and- character-sets-no-excuses/)

Guessing will lead to tears.

<comment><USER27> If it's a mix of UTF-8 and a single-byte encoding, you can guess, because UTF-8 is nice enough to be obvious when you guessed wrong.

The command-line version of ftfy [1] can handle a distinct encoding for each line with a near-zero rate of false positives. (No guarantees the output will be _correct_ , especially if you read it as the wrong single-byte encoding initially, but it will be _no worse_ than what you put in, because it won't replace correct text with incorrect text.)

There is no JS version of ftfy, and there won't be until JS can represent all Unicode characters without surrogates.

[1] [https://github.com/LuminosoInsight/python- ftfy](https://github.com/LuminosoInsight/python-ftfy)

</comment></comment></comment><comment><USER28> 43\. This list matters for all applications that parse or create CSVs.

</comment><comment><USER29> I think many of the problems he lists are because, as he properly notes, a great many people believe "ALL" or "NEVER" are appropriate words to use when considering CSV files (most of his points use one or the other).

But I think it is in all fairness to point out that parsing CSV files for value is a lot better than free form text, even if it's nowhere close to XML.

</comment><comment><USER30> Is probably time someone did a follow up RFC, specifying further useful things. eg:

* how to encode binary data * how to encode NULL's (for transport between databases) * how to specify encoding * (etc)

</comment><comment><USER31> Last time CSVs came up I was downvoted without comment for mentioning HDF5.

I know the CLI tools could use some improvement, and a vim plugin would be nice, but are there other concrete objections?

<comment><USER32> If the requirement is to read CSV files from a third party, then suggesting HDF5 isn't helpful.

If the requirement is to write CSV files to send to a third party, then suggesting HDF5 isn't helpful.

So this limits the usefulness of mentioning HDF5 to when you are both the producer and consumer of the data, but that's the easy case because then you can choose anything up to and including strongly typed binary formats.

<comment><USER31> Thanks - that's fair criticism.

</comment></comment></comment><comment><USER33> I love the fact that if I want to work with CSV from C# I need to use a VisualBasic namespace:

Microsoft.VisualBasic.FileIO.TextFieldParser

see: [http://stackoverflow.com/questions/3507498/reading-csv- files...](http://stackoverflow.com/questions/3507498/reading-csv-files-using- c-sharp)

</comment><comment><USER34> Stop this horrible genre now! As discussed in the recent video programming post ([https://news.ycombinator.com/item?id=13259686](https://news.ycombinator.com/item?id=13259686)), this neither helps the beginner because it is unnecessarily overwhelming, nor the expert because he has already read [https://tools.ietf.org/html/rfc4180](https://tools.ietf.org/html/rfc4180) and is aware of these problems with bad CSV parsers/emitters. In the intermediate case, it could be useful for someone maintaining a CSV parser or emitter who is not aware of a few of these points and wants 100% compatibility with all CSV parsers/emitters in existence, but the list style is confusing grammatically and offers no counterexamples or suggestions for improvement.

With that said, if you are interested in using CSV for encoding and decoding in your own environment, I suggest a new file standard called μTSV.

1\. All μTSV files are UTF-8.

2\. All values are delimited by "\t", and all lines must end with "\n" (including the last one).

3\. If you want to use a "\t" or "\n" character in your value, tough luck--- use JSON.

<comment><AUTHOR> I mostly just thought it was fun to write. Put my pain down on a page, get a little bit of catharsis. I actually had written out some "counterexamples or suggestions" but decided to keep it terse and with the general style of the other similar posts.

I was not aware of the scorn for the style, I just knew I enjoy reading them.

As opposed to your μTSV may I suggest DSV. The Art of Unix Programming makes a strong case for how it's superior to CSV [1]

1\. [http://www.catb.org/esr/writings/taoup/html/ch05s02.html#id2...](http://www.catb.org/esr/writings/taoup/html/ch05s02.html#id2901882)

<comment><USER35> Heh, by coincidence, I had recently written two posts about DSV, and had mentioned the same TAOUP DSV link you gave above:

Processing DSV data (Delimiter-Separated Values) with Python:

[https://jugad2.blogspot.in/2016/11/processing-dsv-data- delim...](https://jugad2.blogspot.in/2016/11/processing-dsv-data-delimiter- separated.html)

The example program in the post lets you use either a "-c delimiter_char" or a "-n delimiter_ASCII_code" to specify the delimiter for your DSV data being processed by it.

and

[xtopdf] Publish DSV data (Delimiter-Separated Values) to PDF:

[https://jugad2.blogspot.in/2016/12/xtopdf-publish-dsv- data-d...](https://jugad2.blogspot.in/2016/12/xtopdf-publish-dsv-data- delimiter.html)

<comment><USER36> You're both wrong. EDN is the best.

[https://github.com/edn-format/edn](https://github.com/edn-format/edn)

<comment><USER35> >You're both wrong.

Bah. How am I wrong when I haven't even claimed DSV to be the best? Just quoting that section in ESR's book does not imply I endorse or agree with it. Logic ...

<comment><USER35> Plus, just providing that EDN link is hardly proof that it is the best ... And further, "best" in many cases is subjective. You want to claim EDN is the best, you lay out an objective proof.

<comment><USER37> I get that the parent response about EDN isn't particularily enticing but you really shoild give it a look.

<comment><USER35> I do intend to. Thank you for the comment though. I had starred EDN on github just now before replying to groovy2shoes, i.e., I took an initial brief look at it. Saw a commit by richhickey (Clojure creator). Since I'm into data / file formats, I will definitely check out EDN.

<comment><USER36> Sorry, my intent (beside bringing up EDN) was to be playful. Tone is often difficult to communicate in writing. I would have stuck in a `:p` if I hadn't been on mobile, but it slipped my mind. My apologies; I did not mean to insult or to otherwise be combative or critical.

I don't have objective proof that it's the best. It's a mix of personal preference and empirical "evidence" based on past success with both EDN and its granddaddy s-expressions for data exchange and serialization.

As for why I personally like EDN so much:

1\. It offers the malleability, flexibility, and extensibility of XML, while being

2\. even more concise than JSON,

3\. straightforward to parse, and

4\. precisely defined in a rigorous and language-independent specification.

Traits 1-3 it shares with the traditional s-expressions of the Lisp family, but in contrast to s-expressions it's specifically designed as a data exchange format rather than as a lexical/syntactic format for programming languages. The reason this matters is because the traditional Lisp reader is itself extensible, and extensions can't be guaranteed to carry over from dialect to dialect or even from implementation to implementation. Many Lisp readers go so far as to even allow the execution of arbitrary code at read time, which is desirable for source code as it enables metaprogramming, but it's not so desirable when you're parsing data from an arbitrary source, due to security concerns.

While EDN does have its roots in Clojure's reader, EDN is _not_ exactly Clojure's external representation. Rather, it's a subset of Clojure's surface syntax in much the same way that JSON is a subset of JavaScript's. Like JSON, EDN works great outside of its "native language" (in fact, I've never used EDN from Clojure itself; I've only used it from Lua, Scheme, Python, C, C++, C#, Java, and JavaScript (not necessarily in that order)).

<comment><USER35> >Sorry, my intent (beside bringing up EDN) was to be playful. Tone is often difficult to communicate in writing.

I understand, and agree it can be difficult. Thanks for clearing that up, and no worries at all. Interesting info, BTW, about EDN. Will reply again in more detail later today with any comments/questions.

<comment><USER35> I'll need some time to check out EDN, so will message you with any comments later, since this thread might expire. Thanks again for letting me know about it.

<comment><USER37> I'm also interested to hear your perspective

<comment><USER35> Cool. Will try to comment before the thread expires, then (not sure what the period is), else will post the points here as a new thread under my id (vram22).

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER38> See also: XML

<comment><USER39> I mean, its a bit apples or oranges.. XML's well-formed rules are, in my experience, fairly universal, hence parsing isn't as much an issue as is poorly constructed XML schemas (and schma-less xml documents as well) I know there are edge cases, but overall XML doesn't have many of the problems specified.

<comment><USER40> XML has a _specification_. There are two well-defined categories of well- formed and ill-formed XML files, and there's exactly one correct way to parse the well-formed ones.

<comment><USER38> You still write a list like

1: it is well formed

2: the person who produces it will fix it when you tell them

3: the encoding is correct

etc.

Even ycombinator's rss feed is only well formed but not fully specified

[http://validator.w3.org/check?verbose=1&uri=https%3A%2F%2Fne...](http://validator.w3.org/check?verbose=1&uri=https%3A%2F%2Fnews.ycombinator.com%2Frss)</comment></comment></comment></comment>


Bomr is a script that automatically removes UTF-8 BOMs from your files - <AUTHOR> https://github.com/jamesqo/bomr <comment><USER2> Just patch the kernel to hide 'em.

</comment><comment><USER3> Does this have a detect-only mode?</comment>


13M Pages of Declassified CIA Documents Posted Online - happy-go-lucky http://motherboard.vice.com/read/13-million-pages-of-declassified-cia-documents-crest-archive-were-just-posted-online <comment><AUTHOR> Tip: use the codeword "keyhole" to browse interesting satellite reconnaissance photos: [https://www.cia.gov/library/readingroom/docs/CIA- RDP78B04560...](https://www.cia.gov/library/readingroom/docs/CIA- RDP78B04560A002000010029-2.pdf)

<comment><USER2> Ref: [https://en.m.wikipedia.org/wiki/Keyhole,_Inc](https://en.m.wikipedia.org/wiki/Keyhole,_Inc) and [https://en.m.wikipedia.org/wiki/Corona_(satellite)](https://en.m.wikipedia.org/wiki/Corona_\(satellite\))

</comment></comment><comment><USER3> The redacted parts in the documents have codes, e.g.:

(b)(1) (b)(3) NatSecAct

What are those codes refering to?

Are only names redacted?

For you who have trouble searching, here's a document I just stumbled on while browsing through the content: [https://www.cia.gov/library/readingroom/docs/0006541525.pdf](https://www.cia.gov/library/readingroom/docs/0006541525.pdf)

It is about interrogation techniques used by USA (power drills were mentioned). Scary stuff.

I wonder, if you train a machine with content from major (all?) news sites, could it then, in theory, predict what was redacted to some degree?

I have an itch to scrape this content. Maybe I shouldn't.

<comment><USER4> They are codes for FOIA exemptions. [http://www.foiadvocates.com/exemptions.html](http://www.foiadvocates.com/exemptions.html)

</comment></comment><comment><USER5> Interesting searches of the cache include "LSD", "hacker", "Martin Luther King", and "psilocybin". After those searches, the resource was suddenly unavailable.

<comment><USER6> It's federal IT. I'm going to guess overload due to extremely resource- inefficient implementation before I guess "uh oh, somebody's curious about MKULTRA, better pull the search".

</comment></comment><comment><USER7> I hope someone downloads the contents quickly, before another administration takes it offline or 'reclassifies' the information.

<comment><USER8> I wonder if that's actually the motivation behind opening it up now rather than weeks or months from now. There are apparently efforts underway at other federal and federally-funded bodies (e.g. EPA, NOAA, NASA, some federally- funded college departments) to get backups of raw data into safe places in case Congress or the Trump administration orders it taken offline or even deleted [1].

[1] [https://www.washingtonpost.com/news/energy- environment/wp/20...](https://www.washingtonpost.com/news/energy- environment/wp/2016/12/13/scientists-are-frantically-copying-u-s-climate-data- fearing-it-might-vanish-under-trump/)

<comment><USER9> Funny that they only worry about that now that Republicans are taking over.

Reportedly, Obama ordered just such a deletion of records.

[http://thehill.com/blogs/congress-blog/homeland- security/268...](http://thehill.com/blogs/congress-blog/homeland- security/268282-dhs-ordered-me-to-scrub-records-of-muslims-with-terror)

<comment><USER10> Is there anything in this article that actually links the deletion to President Obama or explains why the deletion was ordered? Or even any evidence that this happened?

Edit to be clear: I'm not implying the Obama administration did not do things like this but this read more like an anecdote by someone with political leanings. We also have no information about why they were deleted or modified and don't know if the author had access to the full picture. Perhaps they were merged with another record, therefore improving the "haystack" etc.

<comment><USER9> The author made that connection: _" the Obama administration was ordering they be wiped away."_

I haven't seen any followup on this story. Liberal media seems to have ignored it entirely, conservative media reports it without further details.

The author, Philip Haney, has been a vocal critic, appearing on Fox and writing numerous articles, so there's been ample opportunity for the Obama administration to respond if there was indeed an explanation such as you suggest, but I haven't seen any response.

<comment><USER11> I'm sure the Obama administration is just far too busy with finally getting around to the FEMA camps and gun confiscation to answer every little conservative conspiracy claim.

<comment><USER12> Maybe he's too busy closing down Gitmo right?

<comment><USER13> You can't say he didn't try. But the Republican response was so over the top, portraying the prisoners as some sort of supervillains with superpowers, that it never happened. I don't know why the Republicans are so scared of these prisoners.

<comment><USER9> And sharing NSA data (obtained without a warrant) with the Justice Department?

[https://www.eff.org/deeplinks/2017/01/obama-expands- surveill...](https://www.eff.org/deeplinks/2017/01/obama-expands-surveillance- powers-his-way-out)

Can't blame that one on Republicans.

<comment><USER14> If you are actually interested in getting an understanding of what happened, you may want to read the declassified source document instead of reading the EFF's take on an incorrect portion of the NYT article.

<comment><USER9> Or you could share what you know that you think the NYT writer did not.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> Well sort of:

       CIA.gov is Temporarily Unavailable

       CIA.gov is temporarily unavailable. We apologize for the
       inconvenience. Please try again later.

       Posted: Aug 27, 2012 04:31 PM
       Last Updated: Aug 27, 2012 04:31 PM

That was the result of my search 'Nicaraguan contra'

<comment><USER16> Considering they went from about four computers accessing the db, to the Internet, I'm leaning more towards the reddit/HN/whatever-effect :)

Here's to hoping the "someone who got called in to fix it" makes a tremendous work in scaling it.

<comment><USER6> Federal IT. Betting it's chock-full of easy wins for a savvy dev, and those do tend to turn up with a GS number from time to time. They rarely seem to stick, though, which I gather is part of what 18F is trying in a small way to change.

</comment></comment></comment><comment><USER17> They should put it in a torrent

<comment><USER18> Since the landing page required everyone interested to enter a keyword to search, the back-end server can record all query and ip-addrs.

I doubt if WikiLeaks.org use this exact same method to study what document attract traffic|attention. WikiLeaks' new releases are not signed by PGP anymore, I take that as either a Warrant Canary or some kinds of warning.

Of course, if everything packed in a torrent, then there's no way to conduct such a study.

<comment><USER19> You can iterate through all the documents by going to [https://www.cia.gov/library/readingroom/search/site](https://www.cia.gov/library/readingroom/search/site), going through each year, and within each year, going through all the result pages.

</comment></comment></comment><comment><USER20> link: [https://www.cia.gov/library/readingroom/collection/crest-25-...](https://www.cia.gov/library/readingroom/collection/crest-25-year- program-archive)

Search function not working - redirected url is "[https://www.cia.gov/$(SERVE_NS_)/library/readingroom/collect...](https://www.cia.gov/$\(SERVE_NS_\)/library/readingroom/collection/crest-25-year- program-archive") ?

</comment><comment><USER21> CREST has been available to the public for a while now, though not online. ODCI/DI would be interesting to look into but I won't hold my breath finding anything outlining the bastardly of the early 60s. [0]

First terms I looked up, "Budget Hawkeye" for 1963-64. Found some details, hard numbers were redacted. NPIC ^grew^ a lot post '63 judging by the budget requests. Lots of files referencing "Brass Knob" in '63 (daily Cuban flyover). The other interesting details were NPIC monthly memos detailing the daily operations.

Some interesting finds

\- David A. Phillips ('81) [https://www.cia.gov/library/readingroom/document/cia- rdp90-0...](https://www.cia.gov/library/readingroom/document/cia- rdp90-00845r000201240009-0)

\- Howard E. Hunt ('85) [https://www.cia.gov/library/readingroom/document/cia- rdp90-0...](https://www.cia.gov/library/readingroom/document/cia- rdp90-00845r000201240009-0)

\- Jim Garrison ('67) [https://www.cia.gov/library/readingroom/document/cia- rdp90-0...](https://www.cia.gov/library/readingroom/document/cia- rdp90-00965r000402730006-3)

\- Film 'Executive Action' ('73) [https://www.cia.gov/library/readingroom/document/cia- rdp88-0...](https://www.cia.gov/library/readingroom/document/cia- rdp88-01365r000300070003-7)

[0] [https://news.ycombinator.com/item?id=12324273](https://news.ycombinator.com/item?id=12324273)

[1] [https://www.cia.gov/library/readingroom/document/cia- rdp76-0...](https://www.cia.gov/library/readingroom/document/cia- rdp76-00183r000400030140-4)

</comment><comment><USER22> The declassified studies in indigence are an interesting read - I remember one where they documented some of the kooks that wrote to the DCI.

</comment><comment><USER23> The printing hack was pretty nice :)</comment>


Negative Emissions: The chronic complexity of carbon capture - <AUTHOR> https://www.economist.com/briefing/2019/12/05/climate-policy-needs-negative-carbon-dioxide-emissions <comment><USER2> There's something deeply ironic (and more than a little broken) about Drax pushing into CCS. The biomass mentioned in the article, that they burn in huge quantity travels a fair way to be burnt in the UK's largest emitter. Drax _currently,_ with two of the six 660MW turbines still reserved for coal, burns more wood pellets in a year - 13m tonnes in 2016, when only 3 turbines were biomass - than the UK produces wood. Though they now want to convert the last two to gas, not biomass.

So where does this "renewable" source of biomess (sic) come from? Why two thirds of it comes from Enviva, who clearcut mature US wetland natural forests - not from managed forestry.

Drax's renewable claims are 99% empty greenwashing, and they remain part of the climate problem. Drax the Destroyer would be very fitting as their logo.

[https://theecologist.org/2017/apr/10/no-drax-theres- nothing-...](https://theecologist.org/2017/apr/10/no-drax-theres-nothing- sustainable-about-big-biomass)</comment>


Microsoft has built software, but not a Linux distribution - <AUTHOR> http://arstechnica.com/information-technology/2015/09/microsoft-has-built-software-but-not-a-linux-distribution-for-its-software-switches/ <comment><USER2> I can immediately imagine a market for a Microsoft Linux distribution bundled with Microsoft enterprise support and Windows apps and Azure compatibility built in. Might even be a billion dollar market within a few years. I'm surprised Microsoft refuses to consider the possibility. But hell hasn't completely frozen over yet. And pigs aren't being delivered by flying quadcoptor drones.</comment>


Show HN: Enso.fm – Discover new music / Tinder for music - <AUTHOR> http://enso.fm <comment><USER2> Please, please stop making _Login-with-Facebook-only_ apps.

<comment><AUTHOR> We always thought of this as a convenience that you don't have to keep another username and password in your mind. Why do you consider it a problem?</comment></comment>


Quepy - natural language questions to queries in a database query language. - <AUTHOR> https://github.com/machinalis/quepy

<comment><USER2> not bad</comment>


Why I Still Use a Dumb Phone - <AUTHOR> http://greatist.com/happiness/smartphone-dumbphone-why-off-grid <comment><USER2> I still use a dumb phone. The front screen doesn't work, and I just added a piece of duct tape to keep the battery cover on. One of the biggest reasons that I haven't switched is that when I'm walking across campus at work 95% of the people have their face buried in a screen. There is zero eye contact.

A desperate plea to phone manufacturers, even though I realize I'm in the extreme minority. I would snap-purchase a dumb phone that had the following feature list and a $200 price tag:

1\. Crystal clear voice quality.

2\. There is no 2.

I don't want a camera. I don't want a voice recorder. I don't want an mp3 player. I don't want a web browser. I don't want a flashlight. I don't want text messaging or email. This might sound insane, but I actually just want a phone.

<comment><USER3> I would take a dumb phone that didn't call anyone but only texted if it had the advantages of my last dumb phone: week long batter life, drop-ability, and the ability for me to leave it in cabs and have it returned because no one wants it.

I get/make a vanishingly small number of actual voice calls these days.

<comment><USER4> Sounds like you want a Motorola T900, essentially a two way pager. It runs on a single AA.

[http://www.superpager.net/](http://www.superpager.net/)

<comment><USER5> Holy crap... I never knew such a thing existed. I'm dumping my mobile phone tomorrow.

</comment></comment></comment></comment><comment><USER6> The blog post is a type of hipster humble-brag about the non-disadvantages of old tech. It's like posting about how physical books are superior because they don't require power sources and are easy to trade between friends without DRM.

My phone is nearly indestructible.

I've had my HTC Incredible for at least 3 years. I have an ugly case on the back and the same scratch guard over the screen as almost everyone else. And I built a cardholder on the back of it, so the phone is also my wallet.

I will drop it from waist height onto concrete at will, as a demonstration. I'm clumsy and I've dropped it at least 30 times, but it's still 100% functional. No screen cracks or case issues.

(Except for the LED melting the back a bit because I had the flashlight on High Intensity and it turned on in my pocket)

(ps: Why are there icons and a login box covering part of the text of the blog post? The blog was difficult to read because UI elements were in the way.)

</comment><comment><USER7> As usual, it's an enumeration of things that have nothing to do with your phone.

The author paints a picture of a life of serendipity, surprise, getting lost in a quaint cobblestone village, giggling with strangers, discovering a hidden coffee shop, like your life suddenly becomes a romantic comedy on the streets of Prague because your smartphone has been keeping you from bumping into Hugh Grant.

Transcend your phone.

<comment><USER8> I've often wondered why I keep bumping into Hugh Grant.

(I keep the dumb phone for the low cost and, so help me, flip phones still make me feel like Captain Kirk asking to be beamed up.)

</comment></comment><comment><USER9> My smartphone was stolen, then i decide to buy a dumb phone. Then i felt a better life. No waiting for notification, focus on my work. I can forget everything and focus on what i am doing. Email, Twitter, Whatsapp, Instant Message notification on smartphone isn't a good thing for me.

I think my stolen smartphone nearly ruin my life. Previously i really attached to that phone, waiting for "not-yet-known" notification. Thanks God i aware about that thing, and i am really grateful with my dumbphone.

</comment><comment><USER10> I own two android devices, a Defy+ and a Nexus 7, and I have no use for them, and instead use an old gray screen Nokia dumb phone.

The main reasons are:

\- Battery life. My Nokia holds 7-14 days, depending on how much I actively use it.

\- Industrial customers: I can not enter factories with a phone that has a camera.

I thought about using the Defy+ as a navigation aid for sailing, but:

\- The GPS is unable to get a lock, if there is no cell phone connection.

\- The display is not readable in sunlight.

\- The compass is a bad joke.

So the only use case remaining is: A water proof camera that shows only noise in dim light with a display that is not readable in bright light.

I have unpacked, loaded, booted the Nexus 7 once. Since then its collecting dust in the shelf.

</comment><comment><USER11> I honestly don't like articles like these anymore. It's pretty much the same things regurgitated over and over, and typically comes off as holier-than- thou, at least a big portion of the time. Especially once you meet someone like this in person. By "someone like this", I don't mean your grandparents (or parents?) or relatives living out in the woods...

Being poor is about the only good reason, and another $10 - $30 a month wouldn't really break the bank for most people...

If you can't keep a smartphone for ideological reasons, that has nearly nothing to do with the phone, and almost everything to do with you and your personality.

Someone mentioned "owning" their dumb phone as a pro. Pretty sure I own my smartphones. All of them that I've ever had have been mine. I can do whatever I want with it, no questions asked. I don't even have to pay for it if I don't feel like it, with no repercussions other than having my service turned off and a credit ding, just like any other phone. Technologically, I own it even more, as I can do much more with a smartphone in regards to customization than I can with a dumb phone.

<comment><USER12> I hope you don't "own" an iPhone (after Jan 2013) because aren't they supposed to be illegal to unlock unless with carrier permission?

</comment></comment><comment> fsck--off I'm surprised the author didn't mention battery life. My dumb phone can go several days without needed a recharge, it isn't unusual for the battery to last one week or more if I'm in an area with good reception.

<comment><USER13> Indeed. I bought a "dumb" GSM Nokia before flying to Sri Lanka and * _gasp_ * not only was I able to have 2 lines, even two different carriers on the same phone, without too much hassle (something nearly unheard of the U.S.) I just drop a bit of cash at any ol' cell shop and I'm good to go.

Furthermore, I actually _own_ my bloody phone!

</comment></comment><comment><USER14> Just because you have a smart phone, does not mean you automatically have to hook it up to all of your email or "hangouts." It is quite possible to have a phone and use it for your own purposes. To think that the only way to "get lost" is to avoid a smart phone is akin to saying you refuse to ever buy a map of where you are going because that would be too easy.

That said, we avoided them for a time simply because of the cost. Paying upwards of $100 a month is insane for a phone bill. We are currently trying out republic wireless. They've been rather nice so far.

</comment><comment><USER15> These articles seem like bizarre humble brags to me. You want to get lost? Really? That seems absurd, but you can totally just not use the map feature if you want.

Battery life. My phone lasts all day easily. Is it really hard to plug it in next to the bed at night? Not something I have trouble with.

My dumb phone helps me know myself because I don't have angry birds? What does that even mean? Just don't play angry birds?

Waiting for not yet known notification? Really, you just stop living your life and stare at your phone waiting for it to buzz? Sounds like you have bigger issues than your phone.

</comment><comment><USER16> I have a dumb phone, it was the cheapest phone I could buy. I sit behind a computer all day for work; I program at home as a hobby. The ability to contact people is essential, however I don't see the need to insert yet another computer into my life in order to do so.

</comment><comment><USER17> Sometimed I wish I still had a dumb phone. I used to charge my phone once or twice a week, when it was just a basic GSM phone with voice and SMS. Now I have to charge my iPhone pretty much every night. I could probably get away with charging it every 1.5-2 days though running out of battery on the second day could be rather inconvenient. The biggest challenge is when I'm away from electricity for a few days. The irony is that I sometimes have to keep my phone switched off when I'm away on camping trips, so I end up being less connected than in the past.

</comment><comment><USER18> I seem to be giving up on modern technology after being totally enveloped in it for the past decade. Fountain pen, vinyl, feature phone, automatic watch etc.

Maybe it's a side effect of the recession, maybe I don't like what the internet/technology is becoming, but I'm realising that these older products are in many ways better.

A watch that doesn't run on a battery and is a work of art; an album that's collectable and real; a phone that lasts weeks on the same charge; a pen that feels more like I'm using some sort of artistic tool and so on.

</comment><comment><USER19> You know what I miss? Landlines. For a lot of reasons, probably most of them nostalgia.. still, when I'm rich I will only be reachable by landline or letter, that's kind of my goal in life.

</comment><comment><USER20> I have a tablet and a dumb phone. One is for media consumption on the go, I have a keyword in case I want to do something on it, and if I want to make a phone call I use the dumb phone.

The tradeoffs to get a good interactive experience in your pocket (screen size, mainly) negatively impact the functional behavior of a smartphone as an earpiece (for phone calls). I think my preferred all in one device would be a tablet with a headset jack. Keeping the screen on the thing that goes next to the ear makes one end of the deal suffer.

</comment><comment><USER21> Or you could get a smart phone, and not suscribe to social networks on it. If you like getting lost, maybe you should just a) not check your GPS on the phone b) take trips to go to places you've never heard of. The internet can help, but doesn't need to me your source. I respect your decision, but at the end it's all a matter of self-discipline. In H.S. people were face in dumb phones as well...

</comment><comment><USER22> Smart phones can go days without recharge if you disable the data (edge or 3g/4g). I was traveling with my iphone 3gs in France without data and didnt have to recharge for over a week. Only light SMS usage, may be 1 txt a day.

I use my smartphone as an iPod/Audio Books reader, occasional snap of whiteboarding/documents.

</comment><comment><USER23> Unfortunately voice quality is almost entirely a function of the network you're on, which is obviously outside the phone's control. Regardless of how awesome the phone is, call quality will be terrible if it can only suck bits through a straw one at a time.

<comment><USER24> Phone (and network) support for wideband audio (marketed as HD Audio) improves call quality dramatically.

</comment></comment><comment><USER25> I don't have smart phone mainly because I think their recurring costs are a rip off. Besides, I have no need to carry computer in my pocket for just a few hours when I'm away from the desk computer.

</comment><comment><USER26> Just because you phone does email, instant messaging, facebook, twitter, or whatever, you are not required to use them. You can even go out and let the phone home if you want.

</comment><comment><USER27> I also still use a dumb phone (flip phone, no internet, no apps). Here's my logic for not getting a smartphone:

1\. similar to the author of the blog post, I appreciate the idealistic benefits of not having a smartphone. I already spend too much time on the internet, and I feel no desire to spend more time online. Additionally, I've somewhat come to enjoy asking people for directions in cities when I'm lost (particularly when I don't speak the language of the city).

2\. Cost. As of now, the only times when I think "A smartphone would be convenient" is when I'm in a new city and I don't know my way around. I figure this only happens for a couple of weeks per year for me, and the small amount of convenience a smartphone would add is not worth the cost of a phone. I recognize that I would probably get more hooked on Facebook/Twitter/Instagram/whatever and that I might view that as utility if I owned a smartphone, but right now I view that as a time/productivity cost to avoid.

3\. Shit like [http://www.beneaththewaves.net/Projects/Motorola_Is_Listenin...](http://www.beneaththewaves.net/Projects/Motorola_Is_Listening.html) . I have much less idea what sort of things are happening on a smartphone compared to a desktop/laptop, and I don't think I'm satisfied with the current state of mobile security. Also, having a smart phone is essentially moving the sensors closer to yourself (front facing camera, GPS, accelerometer, etc) and leads to a much more intimate form of computing. This just makes the eventual privacy violations even more damaging.

On the other hand, here are some things that made me wish I had a smartphone:

1\. Currently, I wish I could use Simple (formerly Bank Simple) for my banking. However, "to be eligible (for Simple), you must: Own an iOS device with iOS 5.0 or higher, or an Android phone running Android OS 2.3.3 (Gingerbread) or higher.". :(

2\. I'm in the process of writing an application that needs to know if I've completely tasks (real world offline tasks) at given times. Without a smartphone, I can push requests no further than email, which I only see when I check my laptop. Being able to push requests into my pocket would be very useful.

3\. I'm somewhat interested in gathering personal data throughout the day (such as heart rate, how many steps I've walked, etc). A smartphone is pretty much necessary to use any products like Jawbone UP that allow this.

I realize some of my reasons against getting a smartphone directly contradict some of my reasons for wanting a smartphone, but I think that's actually very representative of the mixed bag smartphones are.

<comment><USER26> Don't thing your phone is not spying on you just because you can't program it. A dumb phone is still a computer, it just lacks a touchscreen and a different OS.

<comment><USER27> And all of my Gmail, Facebook, Twitter, etc passwords and traffic... As well as a front facing camera, which could be used for eye tracking.

</comment></comment></comment><comment><USER28> I just bought a prepaid phone and service for about $10 a month.

I can't see how people afford data plans that cost $100+ a month.

</comment><comment><USER29> I can't get lost, too good sense of direction.

So I carry a smart phone anyways.

</comment><comment><USER30> His phone is "dumb" but mobile. So, I'm more _old fashioned_ than he is because my phone is "dumb" but a _wire-line_ _desk set_ and not _mobile_.

Why? Because when I use the phone, I want the rest of my relevant _stuff_ handy -- paper and pen for notes, various papers, and relevant data on my computer.

But I can't make or receive mobile calls? Right.

To receive a call while I am traveling, my voice phone provider, also my ISP, has a message recorder, and then I can get the message via e-mail and download it. Really, for most calls, I would not want to receive a call while traveling because then I would not have materials, desk, etc. to be able to handle the call well. E.g., maybe I'm walking down an isle at a grocery store looking for a one quart or larger jar of sweet pickle relish and get a phone call about something important where I have no desk to take notes, note pad, background info, privacy, etc. No thanks.

For making calls while traveling, mostly I don't want to do that either. My work is to write my software, and I don't travel much. So, when traveling, I would only call for car emergencies, and those are too rare to be worth having a mobile phone, _data plan_ , keeping batteries charged, etc.

And now I'm concerned about privacy: As it is, there is no way the NSA or FBI can track me via my smartphone signal because I have no smartphone! So, the NSA and FBI can't use a smartphone to track me to my local convenience store to buy gas for my car and lawn mower, milk, French vanilla ice cream, and wheat bread, to a local pizza shop for a pickup for dinner, to Sam's Club, Wal-Mart, and A&P for groceries, cat food, etc., to a coin laundry (until I get my Maytag fixed), to Chinese carry-out, to a hardware store to get epoxy resin glue to fix loose joints in wooden furniture, two cycle oil for my grass trimmer, wet-dry sand paper to use for sharpening my French chef's kitchen knife -- really top national security stuff like that!

Mostly my information, especially that relevant to phone calls, is on my computer which is in a mid-tower case on a table at home and usually in front of me and, thus, also not _mobile_. E.g., for a phone number, that's on my computer with other relevant info and easy to find with some of my software. And I have a little very simple software to dial phone numbers -- just write some characters to a COM port starting with AT and have the phone wires going into and out of an old FAX modem (also good for sending letters via FAX to my reps in Congress).

For a mobile phone to have my collection of phone numbers, I would have to copy over the data and, in the future, _sync_ the two locations of my data. Also a _smart phone_ , with maybe something from Microsoft an exception, won't run my most heavily used piece of software that, in particular I use for managing information relevant to phone calls, my favorite text editor. If my favorite editor were emacs, then maybe it would also run on a Linux based smartphone. But, as it is a _smartphone_ is a _computer_ but a pain to use instead of my main computer and, then, a sync issue.

If Microsoft wants to do something to get ahead, then they should make much more clear how to do _syncs_ , backups, restores, including of options on software, and the boot partition, and how to use a home computer as a _server_ for mobile devices, e.g., give good explanations of VPNs and other security issues of opening my main computer to access from mobile devices.

Finally, if I had a smartphone, then it might get stolen, and the data on it might be confidential.

Even worse, if the smartphone had an easy way to get a VPN connection back to my main computer as a server, then maybe all of the data on my main computer -- e.g., my software which should be treated as a trade secret -- could be stolen.

Also, now police seem to like to take smartphones and look through all the data, photographs, etc. there and strain to find excuses to start big legal cases. Also, with my smartphone in their hands and a VPN connection back to my main computer, they would also have my trade secret software.

If I have no smartphone, then police can't grab it and violate my Fourth Amendment rights.

If I had a smartphone and had it visible and if there were some police doing something or other, then the police could go wacko (they basically don't like citizens recording or even watching what they do and know that in practice they can get away with a lot of bullying of citizens they don't like), accuse me of taking their pictures, rough me up, arrest me, search my car, plant drugs in my car and then charge me with possession of drugs, take my smartphone, etc. They could take my car. In _searching_ my car, they could find excuses to trash my car, e.g., cut open the seats. If I had a dog, then they could find an excuse to shoot my dog. Just having a smartphone could trigger some not very nice police to get me into a lot of legal trouble for no more reason than my just having a smartphone. Indeed, likely police are afraid of smartphones.

With no smartphone, police have fewer excuses to bully me around, get their sadistic enjoys, and get me into a lot of legal trouble for no reason at all.

<comment><USER6> 1\. Go wifi-only. (Turn on airplane mode and also wifi). This doubled my battery life.

2\. Text-editor app (I like TED) and Dropbox/Drobo/Async to sync it.

3\. Who still uses paper? That's such an edge-case :p

<comment><USER30> My favorite editor is KEdit. Mostly at my computer, I'm looking at a Web page in Firefox or a text file in KEdit. I use KEdit for notes of wide variety, phone numbers, mailing addresses, etc., any high quality word processing via TeX, typing in source code for software, looking at the log files from my Web site development, etc. Since I use KEdit on Windows XP (hope it works on Windows 7 and Windows Server when I convert to them), maybe it would work on Windows phone.

> 3\. Who still uses paper? That's such an edge-case :p

During a phone call, that's the only way I know to capture info quickly. E.g., if the call is for me to get some info, say, a name, phone-number, or e-mail address, then I write it quickly on paper. Maybe I should type in the info, but I don't use a head set on my phone so only have one hand available for capturing the info.

<comment><USER6> "Shouldering" the phone is an age-old technique.

Who doesn't have notepad++ (or another text editor) autosaving to a dropbox- synced folder these days?

<comment><USER5> Jotting a name and phone number on paper (assuming you have pen and paper handy) is way faster than trying to type it on a 2" x 3" keyboard.

<comment><USER6> I inscribe so infrequently that it takes me some concentration to actually draw each letter with my hand. If I don't focus it becomes an illegible mess. Typing on a full-size keyboard takes no focus at all.

I'm unusually fluent with SlideIt on my tiny 1" x 2" phone keyboard.</comment></comment></comment></comment></comment></comment>


Quantum mathematics could improve web searches - <AUTHOR> http://www.newscientist.com/article/mg20227026.500-quantum-mathematics-could-improve-web-searches.html <comment><USER2> Um, there is no indication of quantum mathematics being used here, instead the article just says this method was used in studying quantum related data sets and "The new method gauges the importance of words in a document based on where they appear, rather than simply on how often they occur."

I've seen that sentence almost verbatim in a dozen different search engine proposals recently.

<comment><USER3> I almost clicked on the title, realized it was probably linkbait, noticed it was New Scientist, then clicked the comments link fully expecting someone to have debunked it. I was not disappointed.

<comment><USER4> Read the article. It claims they have found a better estimate of word importance in a document than tf-idf, which would be very significant. Also it doesn't seem to need text segmentation, which means no need for language- specific tokenizers. The research paper is here (haven't read it yet):

<http://bioinfo2.ugr.es/publi/PRE09.pdf>

<comment><USER2> Hmm I have no background but how can they determine what a 'term' is for the purposes of frequency without some form of tokenization unless they are using an arbitrary maximum length on 'term' sizes and are eliminating small terms.</comment></comment></comment></comment>


Term Sheet Negotiation Tells - <AUTHOR> http://permanentrecord.firstround.com/2011/10/31/term-sheet-negotiation-tells/ <comment><USER2> Funny how "convention", "tradition", "standard terms", and "the way things have always been done" _invariably_ break in VCs' favor. $10k of legal fees is small potatoes but I hate the unbridled arrogance that says taking note of that is somehow being greedy but insisting on its presence is simply being a gentleman.

</comment><comment><USER3> Good post, its important to know what you want and what is important. When I first started talking to VC partners I discovered that they were like any sort of relationship, and like those relationships if they have something that annoys you now, its only going to get worse. So ask yourself early on if you can deal with that.

</comment><comment><USER4> Chris is a good guy - we worked together at Delicious.

I've said it before: entrepreneurs will signal, often loudly and clearly, why not to invest in them.

</comment><comment><USER5> All contracts (including Term Sheets for early financing) are simply a manifestation of the parties' expectations. At the end of the day, relationships are more important. How often are these term sheets enforced anyway?

<comment><USER6>
      >> How often are these term sheets enforced anyway?

Virtually every time. Nothing is ever going to turn out better for the founder than what you see on the term sheet.

The term sheet is handed to the lawyers as a template for the contract. The odds of the contract being more beneficial to the founder than the term sheet is infinitesimal. The odds of there being clauses in the contract making it less beneficial to the founder than the term sheet is significant.

<comment><USER5> OK, but by "enforce", I meant "is the basis of a valid, colorable claim in court that ultimately prevails". IMO, that's the best measure of the paper's value/importance.

</comment></comment></comment><comment><USER7> The Golden Rule - if you wouldn't trust the other party on a handshake, all the paper in the world isn't going to be enough to put the deal in writing.

</comment><comment><USER8> It seems that for many people, getting a VC term sheet is considered a measure of success. Sometimes it seems like that's the goal they are working for.

Often, however, when I see one of these articles of advice from a VC, I become glad that our business is so capital efficient that we shouldn't need outside investment until we're very profitable.

And even then, I'd seek out a good angel, rather than conventional venture capital. (In my career, I've seen enough bad decisions forced onto founders by venture capitalists first hand.)

The one phrase that will never hold sway on me is "that's the way it's always been done." And, frankly, I think it tells you a lot about someone's motivations when they want to put expenses that they control on your balance sheet.

<comment><USER4> I think that VCs seem to look heavily for patterns. Partially because they can't take the time to evaluate too much of the deal (it's already possibly a product or business model that's never existed before) so they tend to read unusual, out-of-pattern stuff as higher, non-understandable risk.

Often "that's the way it's always been done" ends up like that because someone, somewhere got burned. Not an excuse but often the reason.

</comment></comment><comment><USER9> > One founder wanted to negotiate out of having to pay $10K in lawyer fees.

Are these fees for the VC's due diligence investigation, or are they for putting together good paperwork for the company? For the former, the company has just bartered legal services and needs to issue an IRS form 1099 to the VC who has to pay taxes, and may be committing securities fraud by colluding to book the fees as basis cost rather than operating expense.

Anybody know?

<comment><USER10> Neither. The proper way to book merger-and-acquisition-related legal fees (along with other non-recurring professional fees related to the transaction) is as part of the basis cost of aquiring the stock. This increased basis cost is not reflected on the seller's side as increased purchase price.

Indeed, this is how legal fees related to the acquisition of any capital asset would be booked by a company. It's all part of the wonder and horror that is the modern tax code.

<comment><USER9> Indeed. So why do VCs want to stuff the expenses inside the corporation?

<comment><USER10> Because recording an increased basis price means that they need to sell the stock for a higher amount to realize a gain (i.e., a profit). Individual VCs (i.e., angels) would prefer that, b/c it means lower taxable income. However, VC firms prefer the appearance of profitability, so every cost they don't have to take on themselves makes their fund look better.

</comment></comment></comment></comment><comment><USER11> > _Another person didn't want preferred shareholders to have any preferred > rights._

I'm just fine with the idea of VCs getting common stock like everybody else, but I'm not so much in favor of random redefinitions of words. I'd be afraid of the liability to the company when that person started making up his own definitions of things like employment law...</comment>


ORCA – An implementation of RISC-V intended to target FPGAs - <AUTHOR> https://github.com/VectorBlox/orca <comment><USER2> Why would someone use this vs Rocket/BOOM? Is it compatible with cheaper FPGAs?

<comment><AUTHOR> Yes, it is built to be as lightweight as possible. We have used it with wishbone devices on ICE40 chips, which are about as small as they come

<comment><USER3> I've been wondering if one of those chips could fit a CPU.

How do you estimate this sort of stuff? I can estimate software decently, but the penalty for being wrong is seldom so bad. With a hardware project, things actually don't fit and you are out of luck.

Different FPGAs have different sorts of limits, not just 1 different number. Do you go by lines of Verilog/VHDL code? Do you go by some idea of "most crazy operation" and register width? How...?

<comment><USER4> An fpga has finite resources (block rams, pins, logic blocks (lookup table + flop)). If you have a good understanding of the logic you're designing, it's not that hard to estimate resource usage. You just get used to thinking in terms of gates, flops, etc

<comment><AUTHOR> Right, you usually have some idea of how Look up tables an n bit add operation, or an n bit multiplexer etc takes up. You also need to take into account how much memory and multipliers you need since FPGAs have multiplier and ram blocks built in.

</comment></comment></comment></comment></comment><comment><USER5> Presentation that includes size and performance figures:

[http://riscv.org/wp- content/uploads/2016/01/Wed1200-2016-01-...](http://riscv.org/wp- content/uploads/2016/01/Wed1200-2016-01-05-VectorBlox-ORCA-RISC-V-DEMO.pdf)

Summary: smaller and faster than PicoRV32 & Z-scale, slower but smaller than Nios II/f.

</comment><comment><USER6> Any idea why it performs so bad on ice40?

Given that ice40 is the only (unofficially) open source FPGA, I had high hopes for the architecture...

</comment><comment><USER7> Would be nice to see a change log.

<comment><AUTHOR> The biggest feature change is the addition of interrupt support. Also stability and bug fixes.</comment></comment>


Google is a second class citizen on iOS - <AUTHOR> http://bringel.github.com/blog/2013/01/01/google-is-a-second-class-citizen-on-ios/

<comment><USER2> For me this doesn't so much highlight the headline that Google is a second class citizen but rather the inflexibility of iOS compared to Android.

Apple really should take a leaf from android's book and see how integration can be made better on iOS. It strikes me as something they have thought about with the new share dialogue using large icons.</comment>


Browse State-of-the-Art Machine Learning Papers with Code - <AUTHOR> https://paperswithcode.com/sota <comment><USER2> Anybody know any good chatrooms for machine learning? Whether that's IRC, Slack, Discord, whatever. Preferably general chatrooms, as opposed to framework-specific chatrooms, but I'll take anything I can get.

<comment><USER3> Hello, one of the creators here! We just launched a Slack for paperswithcode, feel free to join:

[https://join.slack.com/t/paperswithcode/shared_invite/enQtNT...](https://join.slack.com/t/paperswithcode/shared_invite/enQtNTI3NDE2NjQ0ODM0LTdmNzNjODkwOGY0MjU4YzgzNDZhNGM1YWIzYmZhNzk5MTFkYWU4YWNjN2JjZDhlNjJiYjFkYjYwNjkzYzdiZDk)

</comment></comment><comment><USER4> This seems very similar to GitXiv: [http://www.gitxiv.com](http://www.gitxiv.com) . Maybe there is an opportunity to join forces.

<comment><USER5> GitXiv has been very unprofessionally maintained. Their RSS feed often has junk posts like by a child. They were once good, but that time is long gone. I still subscribe to its feed, but there isn't much to find in it.

</comment></comment><comment><USER6> Are the datasets publicly or easily available as well?

Also would be great to include papers with SOTA results on "tabular" Multivariate datasets, the kind that arise in numerous applications, e.g. EHR/MHR, advertising, finance, etc. In other words, something like the UCI ML Repository datasets (which are mostly "small" but still would be great to know the SOTA models for those), and much larger versions of such datasets — I often see papers applying ML to tabular healthcare datasets but the datasets are often not available.

</comment><comment><USER7> This is very good - next step is to hook something like circle to make sure the code actually compiles and does something. Another idea is to put a flag to praise people who do not put compiled artifacts (like matlab mex).

<comment><USER3> Absolutely, I think that would be amazing!

<comment><USER5> Please be careful to not take on too much. It's not the job of this service to compile and test code.

</comment></comment></comment><comment><USER8> This is a good idea.

I think it should be mandatory to publish your code, dataset (or at least a sample of the verification data), and then the trained model weights if you are publishing a neural network "application" paper.

It's very simple to put it on github etc, and then link it in the paper, disappointing when it doesn't happen.

This problem doesn't just exist for neural networks. There are lots of papers published presenting some algorithm, but they never provide the source you can quickly check or experiment. Certainly now the technology exists to allow things to be more open (ie github).

</comment><comment><USER9> This is awesome! I was just about to ask if there were any resources pointing toward easy-to-recreate machine learning papers.

Is there any way to add some kind of flag/sorting mechanism on this for code that requires massive GPUs/ computing power to reproduce?

</comment><comment><USER10> This is really great. Thank you!

</comment><comment><USER11> I see the data sets listed but no links. Are their any plans to host the datasets?

</comment><comment><USER12> Linking these to colab notebooks would be amazing!

</comment><comment><USER13> This is fantastic! Thank you

</comment><comment><USER14> Does anyone know how the categories are associated with the projects?

<comment><USER3> It's done automatically by looking for category names (and synonyms) in abstracts. This mostly works although it's not 100%. However, if you click on the paper, you can add/remove them in the "Tasks" section.

</comment></comment><comment><USER15> Awesome! Thanks for sharing

</comment><comment><USER16> wow, thanks!

</comment><comment><USER17> thank you</comment>


Ask HN: Who Is Hiring (For Freelance/Part Time Remote Work) March 2011. - <AUTHOR>

As with the last one I made, I figure this would be useful for those of us who work 9-5s for BigCo, Incs, but would like to be involved with startups/other more interesting work in our off-hours. <comment><USER2> I'm always looking for part-timers to help with web development projects on the side. My gigs vary widely, so shoot me an email at email@example.com with a portfolio or resume and we'll talk (:

</comment><comment><USER3> This thread seems to have caught on earlier: <http://news.ycombinator.com/item?id=2270791></comment>


Ask HN: Why are most voice interfaces female? - <AUTHOR> Alexa, Siri, Cortana, 'Ok Google' all have female voices.  Is this random chance, a symptom of something more systematic? <comment><USER2> I worked on automation of some of the PA announcements used on railway stations in the UK, e.g. "The next train from platform 4 is the...". These were recorded sets of words and phrases that could be concatenated to sound quite natural and the content of most were very dull.

Every sound file was recorded in a male version and a female version although the female was used almost all the time. I asked about this and was told the following:

* Women respond equally to male and female voices

* Men instinctively rebel against petty instruction given in a male voice... unless...

* In a threatening situation (fire, football riot etc) orders in a male voice are trusted more

I don't know how true any of this really is when applied in the real world, but maybe the similar studies have been applied to conversational interfaces.

</comment><comment><USER3> There is an _excellent_ , if dated, Nova episode called "Top Gun and Beyond" that addresses this. Navy research showed that the (all male) fighter pilots responded much faster to a female voice, so that's what they put in the voice alerts.

This is a minor point in the program, though. It's really one of my favorite Nova episodes and I recommend it to anyone.

</comment><comment><USER4> Why computer voices are mostly female [http://edition.cnn.com/2011/10/21/tech/innovation/female- com...](http://edition.cnn.com/2011/10/21/tech/innovation/female-computer- voices/index.html) gives a couple of reason, from biology, history and "It's much easier to find a female voice that everyone likes than a male voice that everyone likes"

<comment><AUTHOR> Interesting. Its also interesting how much of a gender construct can come through. Listening to music often times I can't tell male from female but with spoken words it always seems obvious.

<comment><USER5> This likely depends on the music genre and voice effects applied to the vocal track.

Usually, telling if a singer is male or female isn't any more difficult than it is for speakers. While in singing there's considerable overlap in ranges between male and female voice types, the overtones produced by a male singer differ from those produced by a female singer. A C5 sung by a tenor will sound very different from a C5 sung by a soprano.

That said, especially in pop music there's the tendency for male singers to use their falsetto register, which tends to sound 'feminine' or androgynous. Moreover, in pop music you also have an abundant use of voice filters and synthesizers such as the infamous vocoders that make distinguishing voice types more difficult.

</comment></comment></comment><comment><USER6> It may partly be simply because in Star Trek the ship's computer had a female voice. I would guess that many companies do not actually put a great deal of research into selecting the voice, but go with what is familiar without wondering why it was historically done that way and whether or not it was a good idea. When you do research it, there are valid reasons to go with a woman's voice. But I strongly suspect most companies do not actually put substantial resources into the decision.

</comment><comment><USER7> I think the genderedness of this kind of labor explains much of it.... try Google Image Search for "secretary" or "assistant".

[https://www.google.com/search?q=secretary&tbm=isch](https://www.google.com/search?q=secretary&tbm=isch)

[https://www.google.com/search?q=assistant&tbm=isch](https://www.google.com/search?q=assistant&tbm=isch)

</comment><comment><USER8> Wikipedia: [https://en.wikipedia.org/wiki/Bitching_Betty](https://en.wikipedia.org/wiki/Bitching_Betty)

Lots of [citation needed], but interesting historical background on voice warning systems.

</comment><comment><USER9> I imagine it's because of the male tendency to make things they build female. Boats, AI, Cars, etc.

<comment><AUTHOR> lol. my old boat name was 'grunt slayer', I guess it could be female, never really thought about it.

</comment></comment><comment><USER10> Anecdotally, I find higher-pitched voices easier to hear.

<comment><USER11> Anecdotally, I have an instructor whose teaching I can't stand, despite her excellent teaching style, because her voice feels like it's hitting the resonant frequency of my head, and I get a headache after 40 or so minutes.

</comment></comment><comment><USER12> History of the male/female BART voices: [http://www.bart.gov/news/articles/2009/news20090309](http://www.bart.gov/news/articles/2009/news20090309)</comment>


Project 10^100 - <AUTHOR> http://www.project10tothe100.com/

<comment><USER2> This has been posted twenty times this week</comment>


The Dark Side of Cryptography: Kleptography in Black-Box Implementations (2003) - <AUTHOR> http://www.infosecurity-magazine.com/magazine-features/the-dark-side-of-cryptography-kleptography-in/ <comment><USER2> This link discusses a nice example of RSA. There was also a paper in 2014 by Bellare, Paterson, and Rogaway [1] discussing this notion more thoroughly (under the name "algorithm substitution attack").

From that paper: An approach that works against many block cipher modes is to select the IV with for a communication using key K as, say, IV = AES(backdoor_key, K). This is indistinguishable from a random IV due to the security of AES, but someone with the backdoor key can easily compute K = AES_inverse(backdoor_key, IV).

[1] [http://eprint.iacr.org/2014/438.pdf](http://eprint.iacr.org/2014/438.pdf)</comment>


Narrow Roads of Bozo Land: How We Came to Be Governed by Online Mobs - <AUTHOR> https://quillette.com/2018/09/22/narrow-roads-of-bozo-land-how-we-came-to-be-governed-by-online-mobs/ <comment><USER2> TLDR; Rambling tirade against the left and the academy with no clear thesis or proposal. Lousy even for Quillette.

Off-Topic: Most stories about politics.*

*[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)</comment>


Shakespeare firm unveils new theatre - <AUTHOR> http://www.flashnews-online.com/shakespeare-firm-unveils-new-theatre.html

<comment><USER2> yet more spam</comment>


Ask HN: Are GitHub Buttons Gone? - <AUTHOR>

Where there used to be a GitHub "watch" button, I'm getting an error saying "for security reasons, framing is not allowed" (see http://bulbflow.com), and Mark Dotto's GitHub button repo is gone (http://markdotto.github.com/github-buttons/). <comment><USER2> Maybe this?

<https://github.com/markdotto/github-buttons/issues/23>

</comment><comment><USER3> They recently revamped their entire notification system. That probably has something to do with it.</comment>


F# and Haskell, Estranged Cousins - <AUTHOR> http://www.atalasoft.com/cs/blogs/rickm/archive/2009/01/29/f-and-haskell-estranged-cousins.aspx

<comment><USER2> > The most obvious difference between Haskell and F# is somewhat easy to infer > from this graph: object oriented constructs. That is to say, OCaml pioneered > the use of object oriented data structures in functional programming and F# > is it's direct descendant.

Uninsightful. OO in F# is unrelated to OO in OCaml.

</comment><comment><USER3> This reminds me of the distinction between "systems" and "theory" in <http://news.ycombinator.com/item?id=436058>

</comment><comment><USER4> Unfortunately, OCaml only has lightweight threads, and F# isn't very performant in a non-Windows environment. I'm not sure there's really a clear winner here.

<comment><USER5> Take a look at this picture: [http://www.csharp- architect.com/images/MonobenchmarksIncLinu...](http://www.csharp- architect.com/images/MonobenchmarksIncLinux.gif)

Mono performs pretty good in comparison to Microsoft's runtime. The garbage collector still needs some work, but it would not surprise me if Mono becomes very competitive in the near future.

<comment><AUTHOR> That is extremely interesting. I would also like to see a graph of the percentage of APIs implemented and some kind of metric for the amount of testing for each API.

Thanks for the link.

</comment></comment></comment><comment><USER6> Jon Harrop To The Rescue! Hey Jon! Hey.... Ah... disappeared...</comment>


Frank McSherry Responds to Differential Privacy Critics - <AUTHOR> https://github.com/frankmcsherry/blog/blob/master/posts/2016-02-03.md <comment><USER2> mcsherry is hilarious. best line: "I will present a third option: The authors could take a fucking stats class and stop intentionally misleading their readers"</comment>


CALLING OUT MENTIFEX - <AUTHOR> https://dis.tinychan.org/read/prog/1525002170 <comment><AUTHOR> I suppose Hacker News is not interested in Mentifex...</comment>


Moneyball for startups? PG, Fred Wilson, Chris Dixon discuss - <AUTHOR> http://www.splatf.com/2011/09/moneyball-for-tech-startups

<comment><USER2> Angel List has the largest data set of startups, investor profiles and activity on the planet. That makes Nivi and Naval some of the most interesting players in Silicon Valley. I suspect we'll be seeing a very interesting investment thesis coming from them in the next couple of years. I've talked with the Kauffman Foundation, and one of the reasons Kaufman funded them was to have access to that data

Chris Farmer at General Catalyst is pretty open in talking about his data driven investment approach. He is doing a big data play, mining source code repositories, LinkedIn profiles, CrunchBase, etc... He wrote up a little slice of his research here: [http://techcrunch.com/2011/05/25/top-10-vc-firms- investorran...](http://techcrunch.com/2011/05/25/top-10-vc-firms- investorrank/)

I know that SVAngel keeps pretty good stats on their investments, and relies on a good bit of data to make their investments.

A number of top tier VC firms I've talked to this past summer are actively building systems to do more data driven investment.

Brendan Baker's Anatomy of Seed study is one of the best data driven studies of early stage investment I know of. He is working with a grad student to replicate the study to see if his conclusions are reproducible. ref ( [http://www.quora.com/Brendan-Baker/Anatomy-of-Seed-An- Inside...](http://www.quora.com/Brendan-Baker/Anatomy-of-Seed-An-Inside-Look- at-a-1M-Seed-Round) )

BlackBox.vc did the startup genome project, and are actively pursuing early stage investments based on that data. <http://blackbox.vc/>

</comment><comment><USER3> A good entrepreneur is up "at bat" half a dozen times. I don't think there's enough data.

OTOH, as an angel, I've invested in ~50ish startups. Lots of patterns (YC vs non YC, equity seed vs convertible seed, etc) emerge.

<comment><USER4> Can you list some of the patterns you encountered?

</comment></comment><comment><USER5> Well you could say Yuri Milner's YC investment approach is more based on data than anything else. It is on YC's, not on the companies, but meta-heuristics are still heuristics.

</comment><comment><USER6> It does not really matter if you are using hard data like number of previous companies, or soft data like impressions based on personality. In fact, these days I would say success depends more on the VC(s) involved than the founders. Mark Cuban was at least half-right when he compared tech investments to a Ponzi scheme ([http://blogs.wsj.com/venturecapital/2011/08/15/mark-cuban- th...](http://blogs.wsj.com/venturecapital/2011/08/15/mark-cuban-this-tech-vc- bubble-is-like-a-ponzi-scheme/)).

This is, obviously, a generalization -- there are a few companies that stand on their own feet, and have even done so without outside investment. But in the majority of cases it seems like startups are acquired because some kind of "inside" deal is going on. When you think about it, most companies rarely benefit from acquisitions and mergers - instead they slowly die. To list a few: AOL, Bebo, Myspace, Flip, Map Quest, Alta Vista, Netscape, Broadcast.com, Excite, Lycos, Ask Jeeves, Sun Microsystems...

What startups are still going strong post-acquisition? I can think of Youtube off the top of my head...what else? If the parent companies are (generally) not benefiting from acquisitions, why are they happening?

<comment><USER7> Reddit? Whatever became Google maps? Hotmail? Android?

There are plenty of startups which have thrived after acquisition. There are also spectacular failures. M&A is risky, but can have a very high payoff.

</comment></comment><comment><USER8> The interesting thing here is how alike the two scenarios are - despite how prevelent statistics are in baseball, they're still not the be all and end all.

There's a quite story from not too long ago involving the LA Dodgers - for the life of me I can't remember the names or the time period, but it will hopefully come to me when I'm more awake (or someone here might refresh my memory). Every day, an expert in sabermetrics (aka baseball stats) would pprepare a huge load of paperwork for the Dodgers manager ahead of that day's game, and every day the manager would say thanks, wait until he had left the room, then chuck it all in the trash.

Tech investors aren't the only peope who value gut feelings over statistics.

</comment><comment><USER9> Re: predicting it based on qualities of the founders: pg has said determination is very important - based on data from startups, it turned out much more important than anticipated. Therefore, one would expect an objective measure of determination to have (some) predictive power. Maybe not as much as the present YC process; but it would be hard data, and a new perspective which might be revealing. Certainly, scientific at least!

It sounds like a sloppy thing to measure; but Martin Seligman (learned helplessness/learned optimism) has quizzes to measure optimism/pessimism, which have experimentally demonstrated predictive power (and actually have been used to supplement job interviewing, with measurable success). So measuring determination might be possible. There's even checks to prevent/detect cheating. Of course, extra smart candidates with million dollar (VC) motivation may quickly subvert any a test. Still, it would be interesting, intellectually, to see if it did work.

Might also be interesting to ask one of Seligman's students to assess YC candidates for optimism (his definition means that you bounce back from failure with energy - e.g. a pivot). It seems plausible to me that that would also be predictive of startup success (and also predictive of determination - the ability to keep going).

</comment><comment><USER10> It's funny, I would assume pg - as in "a plan for spam" pg - would be supremely interested in running some kind of Bayesian predictive model over YC candidates. Maybe the subtext of that quote was that he tried it and it didn't give any useful data.

An interesting conjecture is that numerically predicting startup success (defined as a high standard deviation of return on investment) might actually be impossible because any venture risky enough to get those kinds of results would fall outside the acceptable error bars of the predictive model. The equivalent in spam filtering would be if you wanted a system to show you only messages that were 99% likely to be spam, but still not spam.

I'm not sure if that actually makes any sense, someone feel free to jump in and tear it to shreds.

</comment><comment><USER11> This data driven approach doesn't even work well for other sports,e.g. soccer. I guess the reason it works so well for baseball is that it's made up of a large number of well defined tiny one to one faceoffs.</comment>


GCHQ Cracks Frank Sidebottom's Codes - <AUTHOR> https://www.bbc.co.uk/news/entertainment-arts-47907370 <comment><USER2> "I'm embarrassed to say, on the very next day Chris's very own code grid was found in the back of his address book. It was almost like Chris Sievey was going, 'There you go, now we've all had our fun, there's the explanation.'"

And they say the universe doesn't have a sense of irony...

</comment><comment><USER3> If Frank Sidebottom is an unfamiliar name, I recommend the short audiobook Frank by Jon Ronson as an entertaining story. Jon played keyboards in his band and is an entertaining writer.

<comment><USER4> oh wow thanks for sharing that! I had never heard of frank but i definitely have heard of jon ronson from his research and look into psychopathy. oh and the men who stare at goats, of course. anyway i must now add yet another line to my ledger of things i really should be looking into

<comment><USER5> The movie Frank is a fictionalized version of the character (written by Ronson), and there was a documentary about the guy as well: [http://www.beingfrankmovie.com/info.html](http://www.beingfrankmovie.com/info.html) I watched the movie and was shocked to learn the movie was based on a real life phenomena.

<comment><USER6> I don't think the film has all that much to do with Frank Sidebottom, beyond the name and the head. It's more a take on _more_ troubled outsider geniuses like Captain Beefheart and Daniel Johnston. As far as I can tell Chris Sievey was maybe quite eccentric and probably a bit frustrating, but Sidebottom was a piece of art he was doing and not an artifact of mental illness like the film's Frank.

</comment></comment></comment></comment><comment><USER7> An example of Frank's original worldview.

Anybody who's only seen the fictional film representation may find it surprising.

[https://youtu.be/yrM6sLx_DXo](https://youtu.be/yrM6sLx_DXo)

</comment><comment><USER8> Any more details on how the encoding worked? Was this just a substitution cypher with some gibberish sprinkled in?

<comment><USER9> Is gibberish called nulls or nonces? I forget.

<comment><USER10> Nonce is short for "number used only once"

<comment><USER11> Once is from the Middle English "anes", meaning "one".

Nonce is from "then anes", meaning "the one".

The "-n" got smooshed into the latter word, to become "nonce", rather like "an ewt" became "newt".

<comment><USER12> Is the term that old? I was told it literally meant N(umber)once.

<comment><USER11> Yes, the construction "for the nonce" goes back to the Middle Ages. The concept of a "nonce word" goes back to 1884, according to this article.

[https://www.dailywritingtips.com/nonce-words-for-the- nonce-a...](https://www.dailywritingtips.com/nonce-words-for-the-nonce-and- nonce/)

</comment></comment></comment></comment></comment></comment><comment><USER13> > I spent a while just looking at them going, 'What could he be saying, what > could this mean?'

> But it was impossible to crack them […]

Confirmed uncrackable. He looked at it Jim, what else was he supposed to do??

</comment><comment><USER14> I didn't know til now that the GCHQ headquarters is a flying saucer like Apple's

</comment><comment><USER15> A couple of quotes from OA with a personal 'translation'

 _" GCHQ told Sullivan that Sidebottom "had a small but dedicated following" among its staff."_

Couple of people do Sidebottom dialogues as an in-joke to the extent that it begins to annoy co-workers.

 _" [After random outer triangles explained] 'Right, we've cracked it during a light-hearted training exercise.'"_

Took a couple of minutes as a starter in a session.

PS: I use a Playfair style grid to jumble up my pass phrases to try to make them less susceptible to rainbow table attack. Am I wasting my time?

<comment><USER16> Forgive me but;

You know you are, you really are.

</comment></comment><comment><USER17> "The country's top codebreakers too seemed flummoxed until Sievey's son Stirling recalled how his dad would get the children to fill an outer row with random symbols, while Sievey would insert real code into the inner row."

Is it really that hard to fool some of the worlds top code breakers, simply by including some random digits?

So a code where every {x} symbol is random, and suddenly you've got an uncrackable code? Surely it cant be that simple?

<comment><USER18> Every {x} could be cracked by normal methods if it was simply added as a rule. So let's assume something just barely more sophisticated and we use a nonrepeating function instead of a fixed {x}. For instance, maybe the nth digit of pi means the next _n mod 3th_ digit is random. How would you even begin to crack this? So imagine "thecodeisx". The first digits of pi are 141592653. So we get a noise pattern of: 1, 1, 1, 2, 0, 2, 0, 2, 0. So the code in deciphered format, but with our noise added, could look something like:

tlhpebcokqodengisxf

And keep in mind that is the code before it's enciphered. Even better, the random characters could be not entirely random but rather weighted to try to bring most characters in the message to a roughly similar frequency. So far as I know the primary tool of code cracking is just plain old frequency analysis. Curious if anybody has any proposals on how this would even be possible to crack.

<comment><USER19> Does code cracking still work in the era of modern cryptography? I thought that cryptosystems like AES and others were essentially impossible to crack if implemented right. What role do codebreakers play these days?

<comment><USER20> The very complex modern approaches to cryptanalysis still borrow from the oldest attacks.

A simple substitution cipher is easily broken by frequency analysis - find the most common letter in the ciphertext and it'll probably be E in the plaintext. Nothing so simple would work today, but we often see vulnerabilities in cryptosystems due to pseudorandom number generators with inadequate entropy. It's the same basic principle (exploiting a lack of randomness to identify patterns in the ciphertext), albeit with vastly more mathematical sophistication. The NSA allegedly took advantage of this principle to deliberately weaken cryptosystems by promoting an intentionally weak PRNG.

[https://en.wikipedia.org/wiki/Dual_EC_DRBG](https://en.wikipedia.org/wiki/Dual_EC_DRBG)

<comment><USER21> frequency analysis works easily if you know the source language, i.e. English in this case, while entropy of the message is critical to cracking, complex approaches are not the only ones which are immune to attacks, for example simple ciphers like using a one time pad is mathematically impossible to crack.

<comment><USER12> The power of a one time pad relies on the inherent entropy of the pad though. So attacks against a real one time pad still need to deal with entropy.</comment></comment></comment></comment></comment></comment>