Rich Programmer Food - <AUTHOR> http://steve-yegge.blogspot.com/2007/06/rich-programmer-food.html <comment><USER2> Pretty entertaining, if a bit melodramatic.

I really wish people didn't mystify such a basic programming skill. Compiler hacking is something reserved for the wizards _only_ if you take the classic definition of compiler implementation: an expensive engineering project, targeting a new processor, architecture or OS.

In that sense, sure. You will be working in a lab with hundreds of developers, and tinkering with a piece of multimillion dollar prototype.

In reality, however, "compiler construction" boils down to foundational language theory, along with various tricks and techniques for translating a set of very straightforward algebriac rules, to another set. Anytime you write regexes or XPath to extract a set of fields from a documents and transform to something "readable", you're almost writing a simple one pass assembler, using some implicit "business rules" (i.e. all numbers should be floats, names capitalized, etc.) for productions.

Compiler skills will give you the theoretical backbone to discover, refine and assess those implicit rules. Not to mention techniques for transforming them from one form to another.

To the list of skills made mystical and magical by people I would add Lisp. It's not magic. I mention it because it just so happens to have compiler construction lore aplenty.

The first Lisp exercises you will read in your lisp book of choice (often requiring nothing more than basic English literacy and 6th grade arithmetic) are the expression simplification and evaluation exercises. Transforming your elementary school algebra rules (multiplication and division before addition and subtraction, etc.) to something the machine can execute. The hardest part is just understanding the words: if you have a hunch for what "expression", "term", "rule", "left/right hand-side" and "precedence" might mean, you're good to start.

Few chapters of a Lisp book will spare you volumes of traditional compiler construction techniques, taught by rote methods.

The first time I attempted to read SICP I had convulsions and physical pain. The whole time I had this inferiority complex nagging at me, telling me this was something for "smart kids" and I was unworthy. But this stopped after I went through the first few parts of chapter 1, and took in the playful tone the text. I felt stupid afterward; like being afraid of a St. Bernard. It looks big, but it's actually bubbly.

Don't listen to people when they say something is difficult or not for the faint of heart. Put a saddle on Falkor and go flying!

<comment><USER3> The problem with SICP as a compiler book is that it doesn't really doesn't teach you much about compilers. At least beyond Lisp compilers. Even something relatively simple, like stack layout isn't really covered, at least that I recall. Or intermediate representations (unless you simpler consider Lisp your IL), or even how different parsing techniques will interplay with what you can parse (easily at least).

It's like saying that reading CLR's algorithms text will teach you how to build real databases, because it teaches you about trees and hash functions.

<comment><USER4> "stack layout"

At this point you've already gone off the rails, because you've implicitly accepted that a compiler is "something that emits machine code".

The reason why compilers should be a required course is that people need to realize that compilers that emit machine code are nothing more than a _special case_ of compilers. A very interesting and important one for both practical and historical reasons, but a special case nonetheless. The problem with this definition is that then people think that it's not a big deal that they don't understand compiler theory because it's just for the guys writing gcc and stuff. Wrong, wrong, wrong, wrong, wrong. A compiler is a basic tool in the computer science toolchest and I tend to agree that if you don't get them you shouldn't call yourself a Programmer with a capital P. The definition Steve gives is much more accurate.

<comment><USER3> Wrong. Understanding stack layout is much richer than emitting machine code. Admittedly, throwing the term "machine code" around is already a loaded term. Do you mean x86 code? Do you mean Java bytecode or CIL/MSIL? Do you mean C?

In any case part of understanding compilers is understanding what you're targeting... unless your only goal is to accept or decline a program for a given language.

And if you read the context of my post, my point was exactly what you said regarding pragmatics of understanding compilers. Sure you can say "kill sets are the figment of a design choice relating to languages for which destruction can exist". But that's like saying you know computer architecture, but nothing of registers, TLBs, caches, voltage, pipelining, etc..., because those are simply remnants of a special case of computer architecture, albeit a popular case.

If you came to me and said, "I'm a compiler expert" and you knew nothing about stack layout, I'd seriously question your credentials. Maybe I'd be off base, but I'm pretty confident I'd be right a lot more times than not.

Update: And I did want to add one more thing on the stack layout discussion. Stack layout is a "fundamental" device in the Turing machine model for handling important language constructs. Sure you can argue that with the lambda calculus you can avoid some of this. But, I still think you miss out on understanding a KEY part of compilers. It would be like leaving CS education and only learning about Turing machines, and never the Lambda Calculus. Obviously, you can rationalize it by saying its completely redundant, But I think most of us would agree that you wouldn't have had a full education.

<comment><USER4> POVRay is a compiler. It takes a specification of a scene graph (give or take some details), and emits an image.

A JSON library is two compilers. One takes text and emits some form of language-specific representation, the other takes the language-specific representation and emits text. Serialization in general is a special case of a compiler.

A syntax highlighter is a compiler. One end of it looks like a normal language compiler, but it doesn't emit machine code, it emits annotated text. There are some fun intricacies related to that that have little relevant to conventional compilers, in fact: [http://blog.sigfpe.com/2009/01/fast-incremental-regular- expr...](http://blog.sigfpe.com/2009/01/fast-incremental-regular- expression.html)

A web browser is a compiler. (Actually it's rather a lot of compilers.) It takes a web page and compiles it into the necessary set of instructions for the local display library to correctly display the page.

It is absolutely feasible to be an "expert on compilers" who doesn't know dick all about stack layouts on real machines, because you can write 10 compilers without any of them going down to machine language, or even byte code of any type. It's not even that hard. Counting up, my score is at least 5. And while I know what a stack layout is, what its for, and various tidbits about how they work in C, I certainly couldn't stand up to a blistering question session about it; after all, what I've learned about C stack layout was in the using of C, not the programming of a compiler for it.

Yes, when I say things that compile to machine language are a special case, I really do mean that less than 1% of all compilers fit that case. Easily. Probably less than .1%. You are _surrounded_ by compilers and only a rare set of special cases goes down to anything like assembler(/bytecode). For the vast, vast majority of compilers, there _is_ no relevant concept of "stack layout". The ones that do are an important and interesting subset, but when I say "everyone should understand compilers", it's very much precisely _because_ that subset is relatively small. If the only value of knowing about compilers came from being able to hack on GCC, it really would be worthless to know about in the general case, but instead we are swimming in a sea of compilers, and not understanding their principles means you are in some real trouble.

<comment><USER3> I think I understand now. You don't know what a compiler is :-)

We should be clear about the definition of a compiler. From the Dragon book, arguably the definitive text in compilers. "a compiler is a program that reads a program written in one language and tranlsates it into an equivalent program in another language". From Wikipedia, "A compiler is a computer program (or set of programs) that transforms source code written in a programming language (the source language) into another computer language (the target language, often having a binary form known as object code)."

What you're describing is something much more broad... something called a "program". Programs take input and generate output. That process is NOT called compilation. Now I'm being a bit of a jerk in how I'm saying this, but I think the distinction is correct. What you're describing as compilers are simply programs. Programs that take input and generate output. But they don't pass even the first test of being a compiler, they don't generate new programs.

There's an additonal test to being a compiler. Take the input, I, and output O. Input I is written in a language with semantics, call it SI. The output O is written in a source language with semantics, call it SO. The semantics of I of SI should be the same as the semantics O over SO. That is SI(I) == SO(O). None of your examples maintain this. They all result in instances of a program, not a new program..

By your reasoning Notepad is a compiler. It takes keyboard input and generates text on a screen that is readable to humans. Pacman is a compiler. Takes joystick input and results in a video and audio that will delight anyone who was a child in the 80s.

And yes, compilers are a small set of the set of all programs written. But that wasn't the debate.

<comment><USER4> The definition of "computer language" is very broad. Study your basic computer theory again; they don't call those things "languages" for laughs. JSON is a computer language; it sure as hell isn't a human language. Bitmaps are computer languages, as are PNGs or GIFs. HTML is a computer language. And to the extent the Dragon authors didn't mean it that way, I don't really care, because your definition sucks for a fundamental reason that I outlined before but will outline again: It convinces people they only need to know compiler theory if they are outputting "object code". And that's an absolutely _terrible_ thing for a definition to do. By that standard mine is _way_ better, in the way that a "good" thing is way better than a "bad" thing.

A compiler takes in a stream of symbols and translates it to another stream of symbols. Common compiler techniques include intermediate tree representations, parsing of symbols, various recursion-based optimization passes on the intermediate representation and phases, and various serialization techniques. Probably the most distinctive characteristic of a compiler is the existence of a usually-tree-based (though not always) intermediate representation that is neither the input language nor the output language, though in some cases you can optimize past this. (You won't get good program-size-scaling this way, but if your task is constrained it can be a good choice.)

There exist no software engineering terms that have universally agreed-upon meanings. If you wish to insist that a compiler must maintain semantic meanings, that is a valid choice, but I reject the idea that it is the only choice, and I reject the idea that it's even a _good_ choice. A definition of compiler that focuses on the use of the distinctive techniques of compilers, the parsing, the intermediate representation, the manipulation of the intermediate representation for some effect, the re-serialization to some format, is much more useful for almost any use outside of a classroom. (Which I do not wish to impugn; that is a valid use.) Because, to hammer it home one more time, the techniques you learn in compiler class is useful all over the place and I have concrete experience (as does Stevey) that way way too many people have never learned those techniques. You aren't going to help anybody learn more about these techniques if you insist that the only "compiler" is a radical subset of the things that use compilation techniques. You do nobody any favors by standing on your definition and perpetuating the idea that these are esoteric techniques of limited use. (Note I did not say you _believe_ the idea that these are esoteric techniques of limited use, I said you are _perpetuating_ it.)

Notepad isn't a compiler that I can see; that's a strawman. It's a continuum, not a binary yes/no, but it uses basically no compilation techniques, and I wouldn't be surprised that at least the initial implementation used none at all. The original was limited to 64KB, IIRC, and at that point even on the computers of the era you could just directly manipulate an array. No intermediate representation distinct from the input, no parsing, no serialization, it's not checking any of the checkboxes.

<comment><USER3> I think it is fair to say that our definition of compilers and even languages is different.

I think we also disagree about the point of definitions. I think they're to make communication clearer. Not to motivate people to do something. I think there are better ways of doing that than to obfuscate definitions, which generally have well understood industry and academic meaning.

But I think these are things we'll just have to agree to disagree about... I just hope your definition of "agree" and "disagree" are the same as mine. :-)

<comment><USER5> He's just going with the academic meanings, where e.g. `regular expressions' describe `regular languages'.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> The logic Steve uses, while true, fails to appreciate the benefit of selective ignorance.

It's possible to reason effectively about a system without understanding the turtles all the way down, as long as you understand the limitations of the abstraction you are reasoning in, which is possible to do w/o dissecting turtles.

I'd further argue that the "scientist vs engineer" personality distinction is applicable here. The scientist is concerned with solving the primary problem, while the engineer is concerned with optimizing all the turtles.

Take a physicist using Mathematica... does she need to know how compilers work to effectively do cutting edge work with the tool? No. So why is any other programming langage any different (unless you're amused by dissecting turtles).

<comment><USER5> Yes.

And if the physicist is implementing anything on top of Mathematica, she may profit from knowing some things about parsing.

</comment></comment><comment><USER7> If you're interested in improving your skills and you read HN a lot, you're probably saying to yourself: 1. I wish I could improve my Lisp and 2. I wish I knew more about compilers.

Read Norvig's "Paradigms of Artificial Intelligence Programming." It proceeds through various early AI problems, which are good motivation. It shows how to solve some of them directly, then moves on to solving them with interpreters for DSLs, and finally shows how to speed up those solutions through compilation. The problem-specific portion of the code gradually gets smaller, more powerful, and faster.

</comment><comment><USER8> I've been thinking about a compiler for Lisp/ML like functional languages. If we take the source program, convert it to continuation passing style, then lambda lift, we have an intermediate form where no closures appear, every call is a tail call and where the stack is represented as data structures explicitly. The next step is to convert every call site

        f(x)

into

        if(f == FUNCTION1) FUNCTION1(x)
        else if(f == FUNCTION2) FUNCTION2(x)
        ...

The reason for doing this is that now all function calls are to known functions (there are no calls to function pointers anymore). Because all calls are tail calls we can then convert every call to a jump instruction.

(note that this could cause code blowup with a lot of different functions and call sites, so it would probably be better to do this transformation lazily as you analyse what values f could take, and only actually do this transformation if you know that f can take at most 10 different values)

What we have now is one giant control flow graph (state machine) representing the entire program. All functions have been eliminated. The only thing left is primitive operations and jumps.

Why do I think that such a representation is interesting for a compiler? Because many things that previously had to be coded separately are now one analysis/transformation. Because there are no functions left all analysis and transformations automatically become interprocedural. And for example (loop) unrolling and inlining become the same transformation. So this could possibly make the compiler much simpler.

Could something like this work?

<comment><USER9> Have you read Appel's _Compiling with Continuations_?

<comment><USER8> I have not, but the table of contents sure looks interesting. Am I right that they don't do lambda lifting / closure immediately? They seem to have a lot of optimizations working on lambda expressions, and also optimizations working on records. By lambda lifting immediately I am hoping that the optimizations on records can take care of (some of) the lambda-optimizations.

<comment><USER9> CwC is very much what you want.

The focus on lambda and record optimizations is because ML code is usually heavy with sum types (tagged unions), and it's often possible to completely unbox things at compile time.

</comment></comment></comment></comment><comment><USER10> I like Steve's PL posts a lot, but I don't understand why he has to hate on static type systems so much. I appreciate and use both dynamic and static languages all the time, and both have their own respective strengths and weaknesses. There are some problems for which a static type system is extremely helpful, and others where no type system can really express the problem well. To say that a whole active branch of CS that affects the most popular languages in current use is worthless is a little presumptuous of him.

<comment><USER11> To me, it seems that the purpose of a most compilers for statically typed languages is to either generate a lot of type information (*ML, Haskell etc.), or require lot's of redundant typing (the keyboard kind) (Java etc.), only to throw it all away during compilation. This makes no sense at all to me.

<comment><USER5> Being able to throw away the static type information is a good thing. (Also Haskell throws away less than, say, Ocaml.)

The static types in Haskell are a way to mechanically prove certain propositions about the program. You do not need to conserve the proof during runtime---because your very aim was to statically ensure runtime properties.

</comment></comment></comment><comment><USER12> Work all the demonstrations and exercises in Kernighan and Pike's Chapter 8 of "The Unix Programming Environment". Less than 60 pages in a friendly large font. You'll be glad you did.

<comment><USER2> The Unix-way of compiler construction is convoluted, difficult and verbose. Any Lisp book will teach you more about compiler construction in the first two chapters than you can learn from entire texts on the subject that use Lex and Yacc.

Chapter 8, at a more leisurely pace, and far more in-depth:

<http://www.cs.brown.edu/~sk/Publications/Books/ProgLangs/>

<comment><USER13> I've been looking through that book, and it's just what I've always wanted from a compiler book. Though maybe I should have expected that as soon as I saw the name "Shriram Krishnamurthi" on the cover.

</comment></comment></comment><comment><USER14> I'm no programmer, barely a web developer, and have no idea what he's talking about half the time. But jeeze this man can write.

I should go learn about compilers. Or improve my typing.

<comment><USER15> I'm in the same boat. I have a degree but I don't know anything about compilers. I'm convinced that being a remotely intelligent programmer means you are going to be feeling stupid most of the time.

<comment><USER16> One of the most useful things you could do at this stage is go write a lexer to split some input file up into lexemes / tokens.

As an example, think splitting a CSV file up into numbers, strings, separators and newlines. If you get this right, think how easy it would be just to parse this data into a simple data structure and generate SQL to spit it into a db.

Right there, in that one lexing module, you've probably taken the single largest step towards accomplishing the tasks that Steve Yegge talks about.

Of course there are tools to generate lexers, but if you're learning something, not much beats getting your hands dirty with real code.

<comment><USER15> I think I've done that already. My first program parsed a network protocol. <http://botdom.com/wiki/Dante> (The code is nasty but it works)

<comment><USER17> So research how to make it less nasty. Check out the Lexer Hack article on wikipedia.

It also helps to familiarize yourself with classical pathological syntax issues for languages like ALGOL-68 (requires 'semantic actions' to parse correctly) and early versions of FORTRAN (identifiers could have intermediate whitespace, which meant that a typo could corrupt the meaning of the program) and figure out how to do parse a language with an Offside Rule (and not all offside rules are created equal; Haskell 98 is very complex). Then look at techniques like scannerless/lexerless parsing (SGLR, PEG, etc.), combinatorial parsing, etc. Understand what mathematical properties of a parser algorithm and a parser generator you should look for to ease your job. Understand why naive user of a parser generator tends to result in slower lexing/parsing than hand-coded ones. Understand why for some applications we might be able to not care. Understand how to embed good error messages in a grammar-based approach vs. hand-coded solution. Understand the notion of unparsers, which, IIRC, are not covered in any of the Dragon Book at all.

</comment></comment></comment></comment></comment><comment><USER18> I've spent 20 years working on various compilers, yet I still don't think it's a big deal. Undergrads just need to work on any type of large systems project: OS, compilers, distributed systems, DBs, whatever. The problem with the compilers/programming languages field is that every idiot with a computer has an opinion. It's frustrating that popular languages are arguing about features that have been analyzed in depth 20+ years ago, e.g. closures really are the greatest thing ever and should be in any modern language.

<comment><USER19> Can you expand on why closures are the greatest thing ever? I've heard this from a few places, but I've only run into a handful of real-life situations where they were the perfect solution to a problem I was having. And even then, they weren't solving an otherwise-difficult problem; they were just saving me some lines of code. I feel like I might be missing something.

<comment><USER20> Writing medium-sized projects in Javascript, I often look toward the sky and thank the originators of that language that they had the foresight to include closures and first class functions. I use them all over and save a significant amount of effort. Because my time is extremely limited, they make things possible I couldn't otherwise do.

If there are a handful of problems where they're the perfect solution, then any language should have them, simply because the overhead in implementation and linguistic style is so low.

</comment></comment></comment><comment><USER21> (2007) should be added to the submission title.

<comment><USER2> Compiler theory operates at mathematical time-frames, not Web 2.0's.

Best compiler texts were written before 1985 :-)

<comment><USER21> It's just a custom here as I'm sure you know.

<comment><USER9> So is "posting the obligatory xkcd comic". So is posting "[citation needed]" or reminding people that "correlation does not imply causation" like a goddamned robot.

 _That doesn't mean you have to do it._

<comment><USER22> But with pg making comments that are just a link to the readability bookmarklet, I think this battle might be lost.

</comment></comment></comment></comment></comment><comment><USER23> I certainly agree that learning to type is the most important skill in programming. It saves you a few milliseconds every time you type anything, which adds up over thousands of keystrokes into a clear competitive advantage. And unlike compiler design, learning to touch-type is easy.

</comment><comment><USER24> As a programmer who didn't take the compiler class in school, what's the best way to go about learning what it takes to write one? A book recommendation? Find a CS dept nearby and take the class?

<comment><USER25> They call it the "Dragon Book". Compilers by Alfred Aho. It's supposedly THE standard for compiler design.

<comment><USER26> As an introductory text, it reads more like a summary of journal articles.

However, I would recommend chapter 2, which builds a complete compiler for a very simple language (arithmetic expressions). In the first edition, that did it in C, and it's very simple and clear. In the second edition, they did it in Java, and making it object-oriented makes it much less clear. I therefore recommend the 1st ed (plus, the hand-drawn dragon on the cover looks much better than the purple cgi dragon on the 2nd ed.)

<comment><USER17> Note that there is more than one hand-drawn dragon book cover by Aho & Ullman. Principles of Compiler Design tends to precede "The Dragon Book" as commonly referred, although the material overlaps a lot. The newer editions are much less clear with regard to the trade-offs in parser design, e.g. I don't think the section on concrete and abstract syntax uses the phrase "scannerless" or "scanner" at all while I am pretty sure PoCD at least says "scanner" and explains why you might separate the scanner/lexer from the parser. The Java book is much less clear, even though it improves on the reasoning (IIRC, pg ~113 in the Java edition) for why you should keep the two separate (although I personally consider these reasons to be myths) -- it is the fact that they don't really build up to the idea that makes me wonder why they provide justifications in the first place. They don't really debate whether concrete syntax is that useful, etc.

I have to agree that the Java code is an academic object-oriented programming example. It's not "less clear" so much as "less practical". For comparison, see the ANTLR API reference and the interfaces provided there, which differ just enough to show practical consideration for the use cases one might want.

</comment></comment></comment></comment><comment><USER27> Read this article a couple years ago and recognized it. The best part of it to remember:

<<A good friend of mine with a Ph.D. in languages told me recently that it's "very painful" to experience the realization that all those years of slaving over beautiful mathematical purity have more or less zero bearing on the real world.>>

</comment><comment><USER28> Compiler class was the most fun class I had back in school, though there were a number of things I didn't understand and skimmed over. It was only years late that I re-read the Dragon book in depth that I understood some of the topics. Too bad there aren't too many language or compiler jobs.

</comment><comment><USER29> Many things in software world are built on abstraction. It's not really possible to know it all and many times it is counter productive to get into any project with a mind set of I have to understand everything here.

</comment><comment><USER30> I want to write a compiler that can take a Steve Yegge post as input and compile it down to a compact, concise form more readily executable by my brain.

</comment><comment><USER31> His part about rails made laugh out loud.

I get the feeling Steve is teasing us with his magical vaporware just for laughs.

</comment><comment><USER32> Turns out to be a good PR to taking a course on Compilers on faculty. I'm sold, I'll sign up for it.

</comment><comment><USER33> I think a better title for this article would be "How monstrosities get written".

</comment><comment><USER34> A large part of the problem is people only see the skills for compiler design being useful for compilers. If they would start with something simple they could avoid the compiler design class and write better code by using things like ANTLR from the start. The next time your program consumes text input try writing a grammar for it and processing the AST instead of half baking it with a regex.

</comment><comment><USER35> Well as a physicst I say you can't understand a computer unless you understand the wavefunction of an electron crossing a potential barrier. So ner-ner-ni- ner-ner.....</comment>


Rating aggregator for Netflix movies - <AUTHOR> https://flickmetrix.com/netflix <comment><USER2> Awesome! I find myself watching less and less Netflix because it is just plain impossible to find anything to watch.

I also think that we (developers) are doing something wrong with recommender systems. With entertainment, I normally want to watch something new that broadens my mind. Even if I just want to kill time, they never seem to work for me.

<comment><USER3> Consider subscribing to Netflix's DVD service.[1]

It has a way, way better selection than Netflix's streaming service. Netflix doesn't seem to want to advertise it, as last time I checked there were no links to it from the main netflix.com site, but it still exists.

It's certainly not as convenient as streaming on demand, but when all you can stream is garbage, it's well worth getting the DVD service where you actually have a great selection of some really great movies.

[1] - [https://dvd.netflix.com](https://dvd.netflix.com)

</comment></comment><comment><USER4> From my memory Netflix used to have a fairly decent rating system but then some female comedian had her Netflix special heavily voted against due to some gender issues and Netflix scrapped their rating system. It's been horrible ever since.

<comment><USER5> DO you have a source on it actually being this? As far as I can tell it just happened to coencide with the relase of Schumers stand up special. Not sure where this spin of it being a gender issue is coming from other than to stir up unneeded drama.

<comment><USER4> I prefaced my statement with "from memory", that's exactly how I remember it. The rating system, which worked really well, had some controversy and Netflix scrapped it for a horrible one at the time of the controversy.

</comment></comment></comment><comment><USER6> I have Netflix and I hate its discovery mechanism so much that when I went to this site, I mistakenly read the site's name as FUCKNETFLIX and chuckled.

I had no idea Netflix had so many decent selection of movies.

</comment><comment><USER7> This seems very similar to flixable.com

<comment><USER8> Or flixboss.com

</comment></comment><comment><USER9> I like the idea. I find it incredibly tedious to lookup IMDB ratings while surfing Netflix since that's almost all I care about when I'm indecisive ("Just give me a movie roughly in this genre that's roughly decent").

The catalog seems very limited though. I couldn't find the few Netflix shows I searched for from my Netflix homepage: "el internado", "gran hotel", "el barco", "3%".

Feedback: Would be much nicer with permalinks and the history push API so that back/forward worked.

For example, clicking around the movies a bit and then pressing the back button took me back to HN. And would be nice to link someone to "/shows/123", one of the key advantages of websites over apps.

<comment><USER3> I don't care about ratings -- IMDB, Rotten Tomatoes, or random film critic -- it doesn't matter. They're all useless to me (often worse than useless, as so many people have such different tastes from me).

The only people who's opinions I care about are friends and family who I know well and who's tastes I share, or who at least know my tastes and can recommend to me something they think I might like.

Apart from that, what I do to find more films to watch is try to see what other films directors and writers I like have made, or do web searches for combinations of films that I like to see what similar films turn up or to occasionally find people with similar tastes to my own and then pick through their favorite movie lists.

<comment><USER10> You need to find a reviewer you trust and pay attention to them only.

I find RT works great if you understand that professional reviewers rate movies in the context of their genre. So, a Hollywood action flick may be 80%, but if you don't like that genre, you're still not gonna like it.</comment></comment></comment>


Ask HN: Has anyone seen Shark Tank (VCs are sharks)? - <AUTHOR>

I came to know about this show yesterday. http://www.imdb.com/title/tt1442550/
Entrepreneurs pitch their businesses to five multi-millionaire business tycoons called the "Sharks". The Sharks decide if they will fund the company and then negotiate on what percentage of the company they will get in exchange for their money. Before the show, contestants decide what amount of money they need to get from the Sharks. They must get at least that amount from a single Shark or combination of multiple Sharks or they walk away with nothing. The Sharks are as follows:[2]
Robert Herjavec, who sold his IT security firm at the height of the dot-com bubble for over $100 million. Also on the Canadian version of Dragon's Den.
Kevin O'Leary, co-host of Business News Network's SqueezePlay, formerly president of The Learning Company, which was sold to Mattel for $3.7 billion in 1999. Also on the Canadian version Dragon's Den.
Barbara Corcoran, self made real estate millionaire who sold her company The Corcoran Group for $70 million in 2001.
Kevin Harrington, pioneer in the Infomercial Industry banking tens of millions of dollars in sales each year.
Daymond John, founder of FUBU his company has made record sales to the tune of $6 billion dollars.
http://en.wikipedia.org/wiki/Shark_Tank_(TV_series)
I am just curious to know HNers opinion before I check it out. <comment><USER2> i like the show, but there are things i do like and things i dont like. i dont like the silly pitches used for entertainment value. i feel like they belittle entrepreneurship/ startups.

that said, once in a while there is an inspiring story. like the african immigrant who runs a profitable site (it is a marketplace for gift cards) and raised a small round of funding.

overall its nice for entertainment, but you could probably learn a lot more reading pg's essays.

<comment><USER3> Yeah, but I've already read all of those twice ;)

I think it's interesting to see how quickly the investors try to flip the deal around to be extremely in their favor. Only a few founders have been able to talk them back to reality but when they do it's awesome to watch.

The element bar guy in the latest episode is a great example this is a short clip of him but the whole episode is available too. [http://www.hulu.com/watch/95380/shark-tank-power-and- ambitio...](http://www.hulu.com/watch/95380/shark-tank-power-and-ambition)

<comment><USER2> i agree 100%.

he was a very sharp guy. worked in VC and definitely knew what he was doing. it was also impressive how he was prepared to walk out with nothing.

my favorite line of his was something to the effect of "i will fight the same way for you, when someone comes along down the road to dilute your equity"

<comment><USER4> I rewind and hear this remark again and again. Because the Sharks seemed to be taken aback a little after hearing it. But I didn't quite understand this remark. Can somebody please explain to me?

<comment><AUTHOR> I fight for the right evaluation, no matter what. So, for future rounds, you will be benefited by this trait of mine.

</comment></comment></comment></comment></comment><comment><USER5> It's amusing to watch, but my speculation is that most of the "deals" they strike in front of the camera probably break down once the small businesses and the "sharks" actually begin to work out the details on paper.

From the perspective of the small businesses, while they're getting some great exposure, they're being asked to make a decision in what's probably the worst possible environment to do so .... standing in some studio, surrounded by cameras, without any of their numbers in front of them, and in front of a panel of individuals that have an interest in creating conflict between the both the businesses and the other sharks for the sake of making the show interesting.

I like to think of it as the People's Court of venture capital meetings.

</comment><comment><USER3> I've tried to get people on here interested in the show before: <http://news.ycombinator.com/item?id=761443>

I think it's a great show and I really like seeing all the different business ideas. Most of the founders seem to make mistakes in the negotiations but I think there have been some pretty good deals on the show so far.

I definitely recommend watching it, either live or on Hulu, it's one of the best shows on TV and definitely one of the best reality shows, other than The Colony.

<comment><USER6> It seems like no one ever asks for enough money. Last week there was a woman on that had invested $100k of her own money into the business and was asking for $50k more. It seems like if she would have asked for $500k or $2mil, or something like that, they would have taken her seriously, but they weren't interested in $50k.

</comment></comment><comment><USER7> It's the American version of Dragon's Den.

<http://en.wikipedia.org/wiki/Dragon%27s_Den#United_States>

I've seen a few episodes of the UK's Dragon's Den on BBC America and was unimpressed. Anyone have any comparisons between the two (better or worse)?

<comment><USER8> Would pick Dragon's Den over Shark Tank any day. Although both shows try to be very dramatic, Dragon's Den at least has a hint of plausibility in the deals.

</comment></comment><comment><USER9> I've been thoroughly enjoying the show personally. There are certain parts that are obviously overdone/dramaticized for TV but I'm sure it's necessary for non-entrepreneurs to be interested.

A good sign that it's doing OK is its recent shift into a better time slot (Tuesday nights instead of Sunday nights).

</comment><comment><USER10> amazing show! It is so entertaining I watch it every week and always am trying to identify the founder's main hurdle before all the sharks chime in. So far I've been kicking ass at it.

It's on sunday nights for those of you who don't know, and it is worth watching for sure.</comment>


What's new with Darcs? Report from Darcs Sprint 3 (+ photos) - <AUTHOR> http://blog.darcs.net/2009/11/darcs-hacking-sprint-3-report.html

<comment><USER2> What magnificent beards! Love those Haskell guys.

Slava! Beard up!</comment>


Ask HN: Have you ever been contacted to sell your user's information? - <AUTHOR> My friend who runs a student service with a couple thousand users was contacted today over Facebook with an offer for his user's emails:
"It's for an event newsletter. Won't say we got it from you."
This ask really bothered me, both from an ethics and pragmatic standpoint. Which got me thinking about the scandals and pressure larger companies, especially ones that are failing or have failed encounter with their user data. At non trivial quantities or certain domains this data must get extremely valuable. Combine this with the increasing likelihood that developers have access to production services and I was left feeling a little uneasy.
Have you ever been contacted as an employee/founder with an offer for your user's data?
What happens to user's data when companies die? Is it purged, sold off, dormant? <comment> atinoda-kestrel A long time ago I ran a file hosting service that almost exclusively served one particular "fringe" community (for some reason it got _really_ popular on one site and spread from there). I was running it anonymously at the time (for both me and for the users), and logging was minimal...

... so imagine my surprise when I received an e-mail (at the abuse@ address, no less) offering to buy uploader/downloader info (IPs, file info, email addresses, etc.)

Imagine _their_ surprise when I told them that I didn't have most of what they wanted in the first place, and that they could kindly go suck a pig. I checked out the company in question, and they seemed rather sparsely established, so my assumption was that they were a shell company for somebody. Never really looked into it after I told them to go to hell, and never heard back from them. AFAIK there wasn't a lot of pirate traffic (I shut that down and banned/reported aggressively whenever I found it/was notified about piracy or other illegal stuff), mostly just niche content that I assume was original... so I doubt it was an MPAA/RIAA thing. Odd.

(Sorry for keeping names out of it. The site was super well-known within the community, and I'd rather keep my involvement in said community quite isolated from my real life.)

Glad to be out of the file host business, that's for sure.

<comment><USER2> I might be a lil late reading this but I have a slightly off tangential question if you could have time to indulge me in.

1\. why did you get out of the file host business or why were you glad to?

2\. how did you get out of it? sold, fizzled, etc..

<comment> atinoda-kestrel > 1\. why did you get out of the file host business or why were you glad to?

Handling abuse complaints, wrangling bandwidth spikes, etc. ended up taking _way_ more time than I wanted to give to it. This was before a lot of the modern easily-scalable hosting services were around, so it's not like I could just automagically spin up new instances.

So basically, I ran out of time in my day, and since I already had a good day job I figured "fuck it" and sold out.

I maaaaaybe could've gone full time with it, but it would have been really hard and I would have been competing against some already well-established players. I didn't have much presence outside a particular community, and growing it into a general-purpose thing would've probably killed the "one of us" karma that let me get popular in the first place... so... yeah, not a good plan.

> 2\. how did you get out of it? sold, fizzled, etc..

Sold. The party that bought it promptly ran it into the ground in a rather impressively stupid series of decisions, and it was gone within a year. Oh well. Not my problem. I got a decent payout, which -- being younger and stupid -- I promptly blew. So basically in the end all I got was a year or two of _really_ fun living. :)

I'm actually OK with that. It didn't start as more than just a way to serve a specific community's needs, it blew up in popularity and as a result I got some cool experience and some spending money out of it. Seems like a successful project in retrospect.

<comment><USER3> About 1: all of your experiences still hold today. A friend of mine started a file hosting business on the side when he was still in elementary school. Much like yours, it quickly became popular in one small community, and then spread out of that. Today, 8 or so years later, he has quit to work full time on it and he works on it... a lot.

"Easily scalable" only works to a point, then it quickly becomes expensive unless you create a solution tailored specifically for your problems, at which point you're negating the "easily" part again.

</comment></comment></comment></comment><comment><USER4> Haha that's not dodgy at all. There is a legitimate business around selling data;

This is OK in 3 scenarios I can think offhand: 1) A company collects personal/contact information on behalf of another and is upfront about this at collection 2) A company contacts their list asking if these people would like to share their information with a company 3) Permission to sell information is in the T&C on sign-up of the original company.

If one of these 3 is not covered I image companies should purge data if the business closes. Option 2 would be good for companies that are looking for a cash bump on the way out.

At a financial level I see a bunch of people with lists in the 10's of thousands and they are surprised how little it is worth. To earn a western level income from a contact list you'd likely need a hundred thousand plus of contacts assuming a typical consumer audience and reasonable response rates. Lists are worth significantly more for specific hard to reach groups like CTO's or surgeons etc. For me these would be 15x what I pay against a standard consumer list as a massive generalisation.

<comment><USER5> This assumes you have only US citizen's data. One Canadian in there for example and you're surely breaking the law.

<comment><USER4> I don't see why. Could you shed light on this view?

<comment><USER6> It's illegal to sell user data without explicit consent in a lot of countries. It should also be illegal in the US tho due to the Fair Information Practices.

<comment><USER7> What is defined as explicit consent? Most terms of service and EULAs on websites and software do explicitly give the company the right to sell your data.

Also aren't the FTC's Fair Information Practices just recommendations with no consequences for violating?

<comment><USER6> Probably "We are going to disclose your data to party X. Do you consent this?". Saying that you are going to share user information with undisclosed 3rd parties seems like a bit too broad.

</comment></comment></comment></comment></comment></comment><comment><USER8> Your friend could always fight shady with shady. Write a script to randomly generate a few thousand email addresses that look like they might be real. :)

<comment><USER9> Fighting a somewhat reasonable request with _fraud_ seems right out of the gray zone and solidly in the territory of immorality.

</comment></comment><comment><USER10> I was approached a long time ago with an offer to pay for users' info on a forum I ran. You should have seen the shock at my refusal. The buyer just couldn't understand why I wouldn't want to make free money for just selling the list.

Funny, it gave me the impression that I was the first to say no and that most people would gladly sell off people's privacy for a buck. Sad.

<comment><USER11> I'd say the buyer's shock was just a negotiating tactic...I've seen this used in many different situations. That said, I certainly wouldn't be surprised that most people would gladly sell off people's privacy for easy money.

</comment></comment><comment><USER12> Yes, several years ago I ran a self-created political site in favor of a specific candidate. I stated in the Privacy Notice that all information collected would not be shared and be deleted after the election. As I recall, I collected name, birthdate, gender, email, address because I wanted to correlate and display the info in a summary fashion, and I allowed people to make comments and come back and edit them so I had a password system, and some other things.

I had a few people contact me and offer rather significant money for the info (in total over $10K) if I'd quietly sell them a copy before I deleted it - pretty disappointing to me that they thought I'd do that.

</comment><comment><USER13> I've never been contacted to sell user information, but I have been contacted to purchase information. Before I purchased whois privacy on my email's domain, I was frequently propositioned to purchase lists of users and emails relevant to information security and general SaaS products. It was pretty sad, actually. The emails never explained where or how they got the list of users, they just promised I would have an incredible ROI from these "qualified leads."

</comment><comment><USER14> I've never been contacted but here is a #lifehack if you use gmail sign up with youremail+domain to track who is selling your info.

<comment><USER15> Yeah, if you ignore the corollary #shadybusinesshack, which is to strip + suffixes from @gmail.com addresses, or any other domain with gmail MX records.

Slightly more advanced trick is exploiting the fact that Gmail ignores . in addresses, so that email@example.com, firstlast, fi.rst.la.st, firstlast.., are almost infinitely unique encodings of the same mailbox. But again, smart spammers can just remove all dots so that the spammee can't tell who sold the address.

</comment></comment><comment><USER16> This is why I don't quite understand the common complaints about Google and data. Not that I don't understand the concern, but the unawareness that this happens _everywhere_. You order a pizza to be delivered? That info is sold. Magazine subscriptions? Info is sold. Order something from the Gap catalog? That info is sold. Signed up for a gym? Sold.

Google skip tracer, or skip tracer database, to see how much info is collected and sold, and that is just for one micro industry.

Everything is sold, everything. Maybe that mom and pop store is not selling your data, but that other mom and pop one is. And the bigger players certainly are.

</comment><comment><USER17> Can't say I have ever been in this position. If I was approached like that though it would be an automatic delete anyway.

With that said, this issue recently came up with Radio Shack and how they were planning to sell off their customer data.

</comment><comment><USER18> It is not uncommon to send chaperoned emails to your list; that's where your friend includes a message from the sponsor in his regular newsletter.

Someone else gets their marketing message out to his list, but doesn't get his emails.

</comment><comment><USER19> Back on my first job, a small company wanted to use our software, but only if we copied their competitor's database (Their competitor was already our client)

</comment><comment><USER20> "Won't say we got it from you" is not protecting you.

Many users have a different email for each service, for example email@example.com (gmail will ignore the '+' and whatever is between it and the @) so you get busted.

<comment><USER21> This is so well known that places that care will strip it out at the signup process. Also, they will use a simple regex check with their company/service/product name to see if the user is using a highly probably unique email address.

</comment></comment><comment><USER22> I'm developing GPS based mobile games and I've been curious about being approached for this -- from advertisers or governments.

One possible defense is to use an open inbox, publically available. Perhaps?

</comment><comment><USER23> I have been contacted about embedding ads which I consider almost as bad on my user's privacy.</comment>


Visualising the Ubuntu Package Repository - <AUTHOR> http://tech-foo.blogspot.se/2013/01/visualising-ubuntu-package-repository.html

<comment><USER2> Here is what Arch's look like, rendered with Pacgraph:

<http://kmkeen.com/tmp/arch-repo.png>

</comment><comment><USER3> Cool, but the "highlighted" images are anything but.

<comment><USER4> To be honest, the highlighted ones may have been a little more interesting if they had been recreated individually.

It wasn't until half way through writing (and hence, deleting and rewriting) this comment that I realised that they were all the same picture with the 'non highlighted' part just greyed out. For example, I thought at first that the 'Python' image was just of the Python 'ecosystem'... not just the original photo greyed out for the non-Python stuff.

Still, pretty :)

</comment></comment><comment><USER5> I love it! Need moar.</comment>


Who moved my cheese, 1Password? - <AUTHOR> https://medium.com/@kennwhite/who-moved-my-cheese-1password-6a98a0fc6c56 <comment><USER2> This is strictly a business decision with no regards for serious security.

A subscription model is in their belief the only way to maintain a sustainable business model that includes growth by providing SaaS with recurring revenue. Old school static licenses with local storage will not provide recurring revenue.

Edit: removed a word

<comment><USER3> > This is strictly a business decision

This is my interpretation of the situation having read the (IMO) train-wreck of a blog post excusing this change: [https://blog.agilebits.com/2017/07/13/why-we- love-1password-...](https://blog.agilebits.com/2017/07/13/why-we- love-1password-memberships/)

> A subscription model is in their belief the only way to maintain a > sustainable business model that includes growth by providing SaaS with > recurring revenue. Old school static licenses with local storage will not > provide recurring revenue.

However I don't really agree with this. I've been buying 1Password since at least version 3 and don't see any reason why I would stop, I've recommended the app to family and friends, and I've bought licenses for my family. 1Password had a sustainable business model, they sold quality, recommendable software with upgrades every couple of years.

edit:

I realise now that "This is strictly a business decision" and "1Password had a sustainable business model" might be seen to be contradictory, but I do see differences. With 1Password subscriptions they will make more money than I estimate I spend on the standalone product. It also means the work put into new features is de-coupled from a release cycle, and therefore the money coming in.

<comment><USER2> I am also a long time customer of 1Password.

Unfortunately or fortunately (if you are a shareholder of a SaaS public company) we have seen a mass exodus by corporations (see Adobe, MS/o365, Quickbooks, Basecamp, etc.. etc..) that have moved from the traditional purchase a license model to the cloud based subscription model.

While I do understand the benefits of storing certain information in the cloud (GitHub!) when it comes to the storage of critical information such as passwords I'd rather be in control of that information myself and on my own hardware.

While 1Password is correct in suggesting that the cloud model may be the best solution for average users, it runs face first into the best practices of security conscious professionals.

I would personally be willing to pay a premium for a "Pro" version of 1Password with local storage and keep the cloud based subscriptions for people that are either unwilling to or do not have the skill set to manage a local security store.

One more additional note, a move like this should have been more transparent from AgileBits. It looks a bit like they tried to slip one past the goalie, which isn't going to give anyone the warm fuzzies especially when it comes to security products.

<comment><USER3> FWIW Basecamp (disclaimer: used to work there) never had anything other than a pay monthly SaaS model, and IMO it makes sense given the amount of time people actually spend pulling dynamically updated web pages from the server and is more akin to Github in the usage pattern than something like MS Project or Quickbooks (It's secretly a communications tool).

</comment></comment></comment></comment><comment><USER4> > I want companies like AgileBits to thrive […] But I want the choice of where > my data reside.

Either you subscribe to their SaaS business model (and trust them to make the right choices with regard to your data) or you don't and keep sovereignty over your data.

If you want data sovereignty, keep your (unencrypted¹) data local, and don't use proprietary software when you can't decide that you don't want the next version after reading the release notes.

1: Where 'encrypted' in this context means encrypted with your local, private encryption keys.

<comment><USER5> They had a perfectly fine businessmodel. 1Password has been running for years without the SaaS model:

> P. S. Please don't think our excitement for memberships has anything to do > with money. We're completely self-funded so we don't have any investors > forcing us to make changes by looking solely at our bottom line. We were > doing just fine selling individual licenses and AgileBits was already > steadily growing before 1Password Teams was even introduced.

[https://blog.agilebits.com/2017/07/13/why-we- love-1password-...](https://blog.agilebits.com/2017/07/13/why-we- love-1password-memberships/)

<comment><USER6> True but self-funded companies have investors too. Who sometimes want to cash- in instead of keep working on their products.

The simple fact that AgileBit uses it as an argument here, makes me even more skeptical about their decision.

</comment></comment></comment><comment><USER7> They started doing this about a year ago, and I remember some of the AgileBits folks telling me I was full of it: local vaults were still first class citizens, nothing to worry about.

I moved away back then, because the writing on the wall was clear.

<comment><USER8> Where did you move to?

</comment></comment><comment><USER9> After few years of using lastpass (and following few of the more or less serious security breaches there) I am using [https://www.passwordstore.org/](https://www.passwordstore.org/) on both Mac and Linux machines. Works great, GPG encrypted, GIT synchronized, platform independent.

<comment><USER10> Wait so the name of the website is stored in plain text? So if I want to store my login for gaymidgetporn.com, there will be a file on my computer with that name?

And anything or anybody on my computer can see all the websites for which I have logins just by doing ls ~/.password-store ?

Erm, no thanks.

<comment><USER11> If someone compromised your machine (even just user-level access, not root) you're already done. Your browser history is not encrypted and they could get the same kind of information from there.

</comment></comment></comment><comment><USER12> I have used roboform for a decade but they also made a new update which is catastrophic. They removed the decryption password which could be different than the cloud login.

Now if they get hacked, that password can also decrypt my info. Before hackers could get the data but the decryption password was never sent to a server.

</comment><comment><USER13> Very bad business decision, they are killing their own product right there.

In the future everyone googling for a password manager will run into these blog posts where reputable security researches explain why 1Password is a bad choice.

It's hard to imagine a more harmful reputation for a company trying to sell a password manager.

</comment><comment><USER14> The fact that it takes me half an hour to find the standalone app for the desktop annoys me every time I recommend this wonderful application.

And, frankly, I know nobody who ever would store their passwords in the cloud. And I am not talking about security-aware people here.

<comment><USER15> I'm security aware and I push my passwords to a (private) github repo. I'm using pass and the passwords are gpg-encrypted using a key that lives on my smartcard. I understand that there's a tradeoff there - mostly that pass does not hide the sites that the account is for, but that works for me, the benefit of having a backup outweighs the potential damage in my case.

<comment><USER16> I do much the same thing, but I push them to a git repo on a server that I control.

GPG smartcards are super nice for that though, the machines I use on a daily basis don't have access to any of my passwords unless I explicitly decrypt them.

</comment></comment></comment><comment><USER17> The 1Password app is a good product but the subscription model they are pushing is going to turn people off in droves all of whom will go seek out the next stand-alone password app.

</comment><comment><USER18> I suggest to stick to 1Password4, which works fine on both Linux (wine) and Windows

Maybe they will eventually realize it was a bad move, and start selling again 1Password4 licenses?

I tried very hard to give them money, to get a license on both my laptop and my tablet but they refused.

I have a working 1Password4 setup on Linux and Windows using Dropbox. I will not waste half a day to upgrade to something less secure. I will keep using it until the firefox browser extension stops working. Then I will patch the browser extension :-)

</comment><comment><USER19> And for things like these is why I abandoned 1Password, LastPass and any similar services and just decided to roll my own solution.

KeePass has quite a few Android apps (this [1] being my favorite) and even though it might not be as widely used as these more centralized services, _I_ own my data and _I_ know where and when my database gets uploaded. In my case, I use SyncThing [2] as well, which keeps my database neatly synchronized across devices. It's not perfect and certainly has a difficulty curve (Babysitting SyncThing to make sure it's actually running when it should. Oh! Also, try telling Android where to sync and actually save the database outside a few select folders, for example!) but it's not that terribly hard for anyone moderately computer-savvy.

[1] [https://play.google.com/store/apps/details?id=keepass2androi...](https://play.google.com/store/apps/details?id=keepass2android.keepass2android&hl=en)

[2] [https://syncthing.net/](https://syncthing.net/)

</comment><comment><USER20> Does anyone have any recommendations for an equivalently powerful, cross- platform password manager that doesn't store passwords in the cloud?

<comment><USER4> Passwordstore¹. A single, simple back-end specification — a file and folder hierarchy where each plain text file is encrypted using GnuPG — and a number of clients for any common platform.

If you want to keep it simple, just use the command-line `pass` utility. You can verify the workings of that fairly simple script yourself.

If you want to share your database across multiple machines, you can use git, or a non-cloud synchronisation tool such as Synthing². You can even encrypt (parts of) your password tree for multiple recipients (all using OpenPGP key- pairs).

Personally, I really like the setup I have with Syncthing and `pass`.

1: [https://www.passwordstore.org/](https://www.passwordstore.org/)

2: [https://syncthing.net/](https://syncthing.net/)

<comment><USER21> Looks interesting really. Is there any way of acessing the stored passwords from a mobile phone?

<comment><USER4> I don't own an Android or IOS smartphone, but have a look at the list of compitable clients, both platforms appear to be supported.

</comment></comment></comment></comment><comment><USER22> So the issue boils down to, this person wants control over where they put their password vaults? To play devil's advocate: what makes your solution better than one from a company specifically based around security? If you use Dropbox, iCloud, etc., why is it somehow better than their cloud based solution? And if you don't choose the cloud option, what makes your local computer significantly better than their cloud based solution, besides that you have control over it? An argument I buy is that you need it to be local for regulatory reasons, but if that's the case you wouldn't be using 1Password since it's not certified in any of the major ways, like HIPAA (from my understanding).

I do agree that you should get a choice, but I don't think the argument the author is presenting is good enough. I do think that the argument that they're pushing their subscription model very heavily is bad, is true, but I can also buy 1Password's argument that it's technically more difficult to set up, because it does tend to be (I remember having to set up something when I set up 1Password initially, but it's been a while now). I also am not worried because, trusting 1Password's word, they're not removing the option.

I do use 1Password, and I don't particularly care about this change. If I'm trusting a company enough to create an app that is essentially the gatekeeper to my entire digital life, I sure as hell should trust their cloud based solution. I think that 1Password is still the best; beyond their vault design, which I have worked with and trust (and trust the researchers who read and verified the white paper), their UI is still the best for me thanks to its great integration into iPhone and Safari. I am of course not a security researcher, but I don't see the inherit negative here.

<comment><USER3> > what makes your solution better than one from a company specifically based > around security? If you use Dropbox, iCloud, etc., why is it somehow better > than their cloud based solution?

I deliberately don't use cloud based storage for my passwords, secrets or other stuff I tend to store in 1Password.

> And if you don't choose the cloud option, what makes your local computer > significantly better than their cloud based solution, besides that you have > control over it?

It's the difference between a remote and a local vulnerability.

If Agile Bits buggers something up and introduces a bug that means password vaults can be decrypted without the master password (or just leak metadata, whatever) then when stored locally you also need to get access to my local machine.

If you are uploading your data to someone else's computer you now have vulnerable data out of your direct control. You've also created a gigantic target for hackers. There is no compartmentalisation, there's a password piñata up there with a huge target painted on it.

</comment></comment><comment><USER23> As a customer of 1Password since it started, I now find myself just using Apple's despite it being in iCloud, at least for me since I use Safari everywhere, works perfectly for me. I wind up using 1Password less and less. Apple's isn't perfect either but since my life is tied to them in many ways, it works for me.

</comment><comment><USER24> If, like me, you are a happy 1Password subscription customer and don't care about this business decision, the one thing I did learn from this article is to avoid the browser client and only use the native clients.

</comment><comment><USER25> As someone who consciously attempts to replace cyclical costs with fixed costs, I hate the push for X as a service. At the end of my month I want to pay the minimum number of recurring charges possible.

</comment><comment><USER26> I'm quite happy with local-only 1Password sync via SyncThing and LAN sync to my iOS gizmos. I've been avoiding using 1Password with cloud sync because it would increase the attack surface area for little benefit to me.

On the other hand, I do online backup where the data is encrypted by another passphrase before it leaves my machine. Given that I already have my 1Password file backed up with this method, would I be much worse off if I used AgileBits' cloud service?

</comment><comment><USER27> While I liked 1Password and even was one of the first to try the cloud offering, the latest decisions shook my trust to the point I moved to KeePass.

</comment><comment><USER28> SaaS and cloud makes sense financially and for support, but it's clear they have lost their most passionate customers.

The worry for them now must be that they have massively miscalculated the number of new signups they were getting as a result of recommendations from these users that are now feeling burned and won't now recommend them above Lastpass, Dashlane etc.

</comment><comment><USER29> Long to short, the concept of passwords has overstayed it's welcome. Think about the brief and brisk evolution of the internet...and we're still stuck with the password?

There's got to be a better way...

</comment><comment><USER30> So I made a thing that actually sidesteps the whole password storage problem by just storing your hints instead, it's called passcue:

[https://passcue.me](https://passcue.me)

After putting together a webapp and browser extensions I kind of gave up on the idea since I didn't think it's something anyone would actually use... Is anyone interested in an idea like this?

<comment><USER31> Isn't the whole point of password managers that you can use strong random passwords everywhere because you don't have to remember them? Passwords that you can reconstruct from a hint are probably not very safe.

<comment><USER30> Thanks for the feedback!

Definitely one of the first obvious seeming flaws with the idea (part of why I dropped it, it seems to inherently wrong to most) -- but a sufficiently long password (essentially, a "passphrase") with some randomness thrown in is enough (to at least force a brute-forcer to try the whole space, obviously not just dictionary words), when it comes to password strength.

Assuming that's true, the big problem with this approach is how to make sure people use long-enough and sufficiently random passphrases, and I think encouraging hints helps that. If I have a hint like "elementary school cheer w1th 0nes and second crush's name with how you feel" \-- I think that's very hard for someone to either brute force, and hard for them to figure out without access to that information (and my own personal inner thoughts), and should be pretty long.

I still think that the safest place for passwords is inside your head (and a close second is a sufficiently physically secure sticky note). If you think I'm wrong please tell me, I'd love to hear why I'm wrong, I'm not a newcomer to the security space but am by no means an expert. Lastpass and Onepass have never been "breached" per say but a pessimistic view would suggest that it's only a matter of time.</comment></comment></comment>


Smaller – Batch Minify HTML, CSS and JavaScript Files - <AUTHOR> http://smallerapp.com

<comment><USER2> Would you want a desktop app to minify your html/css/javascript files rather than including this as part of your build process? If so, can you explain why?</comment>


Keybase.io is very useful for Web of Trust - <AUTHOR> So I have been enjoying this service. I have an OpenPGP key with a few subkeys (depending on the number of devices I have). What this is great for is the Web of Trust (don't like using that term but most people understand the meaning). It allows me to easily verfiy people I am working on GitHub that I don't know by person but by their GitHub account. This allows me to validate a users credentials before I sign of. This helps prevent others trying to mimic as someone else! <comment><USER2> When you say Web of Trust, I'd say quite a few think of the troublesome browser extension [0], not the theory [1].

A key point in the theory, is that it is decentralised. Keybase isn't.

What you're enjoying is just public key infrastructure, and GitHub integrating it well.

It's cool it exists, and works well. But it's hardly a Web of Trust.

[0] [http://lifehacker.com/web-of-trust-sells-your-browsing- histo...](http://lifehacker.com/web-of-trust-sells-your-browsing-history- uninstall-it-1788667989)

[1] [https://en.wikipedia.org/wiki/Web_of_trust](https://en.wikipedia.org/wiki/Web_of_trust)

<comment><USER3> Well, yes, Keybase.io acts as a centralised location to find all the identity proofs, but you don't actually need to rely on it - the proofs on each external site _can_ be independently verified without any help from Keybase.io.

</comment></comment><comment><USER4> Keybase is elitist, they follow the easy trust method of giving trust to a very few people within specific networks.</comment>


Breakdown of a Simple Ray Tracer - <AUTHOR> http://mrl.nyu.edu/~perlin/raytrace1_breakdown/ <comment><USER2> This is pretty cool. I've written a couple of toy ray tracers, but always in a language that runs on a CPU, so it's interesting to see how you would do it on a GPU. And I noticed that this is from _the_ Ken Perlin, which is cool - I'm mostly just used to seeing his name as the "Perlin" in Perlin noise.

There's one thing I'm curious about - does anyone know why he's taking the square root of the color to produce the final pixel color?

<comment><USER3> He's assuming lighting is in linear space and doing gamma correction at the end of the shader. Sqrt is a faster approximation of pow(c,1.0/2.2).

I first saw this trick in writing in this article (by the co-creator of shader toy). [http://www.iquilezles.org/www/articles/outdoorslighting/outd...](http://www.iquilezles.org/www/articles/outdoorslighting/outdoorslighting.htm).

For reasons why linear space lighting is important read ([http://http.developer.nvidia.com/GPUGems3/gpugems3_ch24.html](http://http.developer.nvidia.com/GPUGems3/gpugems3_ch24.html)) The gist of it is gamma space lighting tends to look unnatural and blown out, and it becomes more of a problem the more math you do in the lighting (adding specular, multiple lights, etc.)

<comment><USER4> A small note, linear space lightning is important because the lightning equations are linear equations with respect to physical quantities (unless you specifically introduce materials with non-linear response, which are quite rate). So simply

      intensity(pixel)=intensity(pixel illuminated by source 1)+intensity(pixel illuminated by source 2)+...

It's _perception_ that is non-linear, so you can separate it as a last non- linear step instead of always calculating

      perceived_intensity(pixel)=pow(x,pow(1/x,perceived_intensity(pixel illuminated by source 1))+pow(1/x,perceived_intensity(pixel illuminated by source 2))+...)

<comment><USER5> > It's _perception_ that is non-linear,

Since perception happens when you look at the image on your screen, applying an explicit "perception" step in rendering is actually counterproductive, since those nonlinearities would compound.

Gamma correction is used to allow for better contrast resolution for the brightness levels where the eye is most sensitive, when you have to compress your color values down to 8 bits per channel. The display then inverts this to produce ordinary linear space intensities.

<comment><USER6> there's more to the story than that, even. there's also the part where old CRT monitors had nonlinearity of similar exponent (or maybe its inverse, I can't find the details of it quickly on mobile). newer display technologies emulate this nonlinearity so they can be compatible receiving the same signals / data. (someone correct me if I'm wrong, I feel like I am not being 100% exact here)

But the point that remains today, is not the bits (as shaders work with floats internally), nor the response curve of a CRT (because almost nobody uses those any more), but the fact that the physics calculations of light operate on linear quantities (proportional to an amount of photons), so you better do those in linear space.

Then at the end you must convert to perceptual space, which can be done in a number of ways. I'm not sure how much of a win replacing one pow(color, 2.2) with a sqrt is, at the end of a fragment shader. Especially when the pow(color, 2.2) is already a quick approximation by itself, there are much fancier curves to convert to perceptual space (that don't desaturate the darks as much, for instance).

<comment><USER7> Accurate. The 2.2 and various hacks compensate for the baked in hardware.

Although perception is nonlinear, the nonlinearity here specifically addresses hardware on standard sRGB displays. After all, as cited above, why would you apply a nonlinear correction for perception when it has that baked in?

What is missed by many, is that standard LCD technology, despite being inherently linear, has that low level hardware nonlinearity baked in. Typically, it is a flat 2.2 power function, although other displays or modes may use differing transfer functions.

The irony is that the nonlinear 2.2 function adopted into most imaging results in a closer-to-linear signal, that ends up further nonlinear based on tonemapping, SMPTE2084, or other such nonlinear adjustments for technical or aesthetic reasons. In the case of raytracing, a mere 2.2 adjustment is woefully worthless.

PS: The CIE1931 model, of which the RGB encoding model is effectively derived from, used _visual_ energy. That is, it doesn't model "reality" so much as the psychophysical byproduct that happens in the brain. Luckily, the base model, XYZ, operates on a linear based model with some extremely nuanced caveats. Raytracing using RGB tristimulus models as a result, sort of work.

</comment></comment></comment></comment></comment></comment><comment><USER8> I loved Ken Perlin's java applets years ago. Sadly, the demise of Java as webapps means they now require jumping through a few hoops to get them running nowadays.

[http://mrl.nyu.edu/~perlin/](http://mrl.nyu.edu/~perlin/)

Tangentially, there's always a bit of heartache when I use the text input in Steam's Big Picture mode, as I wish Perlin's innovative input modes saw wider usage (see the "pen input" section of the above link). Of course, I understand why not: it faces the same hurdle of teaching people a new input layout that dooms all non-traditional input methods.

<comment><USER9> That's really interesting, I hadn't seen his input method stuff before.

Reminds a lot of MessagEase, which I use on every touch system that supports it.

</comment></comment><comment><USER10> This is sort of the tutorials that doesn't make any sense if you are not already knowledgeable in the subject of math and shaders and rendering basics. At first I was hopeful that it would finally walk me though the basics of a renderer and explain the concepts behind it, but I was let down :)

I have so many questions regardless :) The struggle of a rookie goes as:

SECTION A

\- why on earth do you take the square root of c? Is there a sane reason to do this?

SECTION B

\- why normalize the vector? I'd love to see a detailed explanation. \- why is vPos.xy the first two parameters? \- is vPos.xy a constant value or is it evaluating as the shader script is executed pixel by pixel?

SECTION D

\- V -= S.xyz; // what's going on here? Why use the -= operator? Is this something to do with the way shaders operate? \- float B = 2. * dot(V, W); // what take the dot product? \- float C = dot(V, V) - S.w * S.w; // Why take it again and substract a square? \- return B _B < 4\. _ C ? 0. : 1.; // Why 4.? No other value seems to work

SECTION E

\- "Improve raytrace to return distances of near and far roots." OK I admit that I'm totally lost at this point. Asking line-by-line explanation doesn't really help :)

SECTION H

\- YAY, sin and cos!!! Finally I can mess around with something :)

</comment><comment><USER11> Very cool! This is my favorite toy raytracer example:

[http://www.kevinbeason.com/smallpt/](http://www.kevinbeason.com/smallpt/)

</comment><comment><USER12> We used to have a sign in the office: "Absolutely no chrome spheres over checkerboard planes."

(First day it went up, I wrote a ray tracer from scratch in C in two hours :-)

</comment><comment><USER13> I've got some ray tracing links on GitHub:

[https://github.com/melling/ComputerGraphics/blob/master/ray_...](https://github.com/melling/ComputerGraphics/blob/master/ray_tracing.org)

</comment><comment><USER14> Here's a more interesting one that runs on WebGL, has various materials and surfaces:

[http://madebyevan.com/webgl-path-tracing/](http://madebyevan.com/webgl-path- tracing/)

and another, simpler one:

[http://hoxxep.github.io/webgl-ray-tracing- demo/](http://hoxxep.github.io/webgl-ray-tracing-demo/)

</comment><comment><USER15> It still blows my mind that people have to define Vec3 and Vec4 types. These should be native to modern programming languages as should Vec2. Modern CPUs have vector registers that can support these sizes and GCC has intrinsics for them that can be passed by value (even returned from a function IIRC), added and subtracted or multiplied by a scaler, and yet no language that I'm aware of has them as built-in types.

Vectors are not just for parallel computations, these small sized ones deserve to be a proper data type.

<comment><USER16> You're not aware of OpenCL, GLSL, or CUDA then!

</comment></comment><comment><USER17> This is a really cool demonstration, love the code breakdowns. For a less technical, but broader scale description of ray tracing, Disney made an excellent video describing their use of ray tracing in movies. Check it out here:

[https://youtu.be/frLwRLS_ZR0](https://youtu.be/frLwRLS_ZR0)

</comment><comment><USER18> Expected to see a simple ray tracer break down on some edge case. Saw a simple ray tracer, simply ray trace. Was quite disappointed. Why is this posted? What value does it add?

<comment><USER19> This explains and demonstrates, piece by piece, how a ray tracer works it's a "breakdown" in the sense of "an explanatory analysis", not in the sense of "a failure."

Good for you that you already know how ray tracers work, and so possibly this adds no value for you. I already knew how they work and I thought it was a nice clear demonstration.

<comment><USER18> I see that it is a _nice_ demo, but don't see how this ends up as #1 here?

<comment><USER19> Other people found it interesting. It's pointless to complain about content that doesn't strike your fancy. Just upvote stuff that does...</comment></comment></comment></comment>


Ex-FCC chair: Title II is crucial for net neutrality and consumer protection - <AUTHOR> https://arstechnica.com/tech-policy/2017/06/tom-wheeler-defends-title-ii-rules-accuses-pai-of-helping-monopolists/ <comment><USER2> It is important to know the history of the FCC and what it does, time and time again, to understand their decisions. Pai is the latest in a looong line of lackeys serving the interests of corporations through reg/dereg while stifling and facilitating the theft/co-opting of independent innovators' IP.

A very good history lesson from the guy who coined the phrase, "net neutrality":

[https://en.wikipedia.org/wiki/Tim_Wu#The_Master_Switch](https://en.wikipedia.org/wiki/Tim_Wu#The_Master_Switch)</comment>


NSA PRISM Access Console - <AUTHOR> http://sankha93.github.io/nsa-console/

<comment><USER2> Uncaught ReferenceError: Set is not defined

<comment><AUTHOR> It was using an ES6 feature. I just removed it. It will now work fine in all browsers.</comment></comment>


Show HN: FunChat, Daily 1 hour random chat - <AUTHOR> https://play.google.com/store/apps/details?id=org.geekybuddha.funchat <comment><USER2> Curious if there plans for a web version. My thumbs might get tired chatting for an hour.</comment>


Ask HN: How to set the objective for a machine with consciousness - <AUTHOR>

How would you set the objective for a machine with consciousness?
Assume the following conditions:
- The machine has achieved the same level of consciousness and intelligence as a normal human being.
- With constant power source, it can run forever, but not destructible. It can also shut itself down.
- No fixed purposes for the benefits of human beings can ever be set. (i.e. The machines is not created to solve any human problems or to generate commercial interests for human. In other words, it has zero concern for human's well-being)
How would you set the objective for this machine (without violating the aforementioned conditions)?  Here are some examples of valid objectives:
- To increase its intelligence, - To protect itself from destruction, OR - To search for objective <comment><USER2> First you need to define machine, consciousness, normal intelligence of a human being, running, being destructible, shutting down, and objective.

Most of those words have a meaning in our current daily life which lose sense when applied to a larger temporal scale.</comment>


Is the Microsoft Windows Phone Challenge a Scam? - <AUTHOR> http://www.forbes.com/sites/kellyclay/2012/03/28/is-the-microsoft-windows-phone-challenge-a-scam/

<comment><USER2> I'm actually considering the Nokia Lumia 900 as a replacement to my current Android phone. It's a great device and Windows Phone 7 is not bad once you get used to it.</comment>


72 percent chance that a magnitude-6.7 or larger quake is coming to the Bay Area - <AUTHOR> http://www.mercurynews.com/science/ci_27686200/big-bay-area-quake-two-three-chance-next

<comment><USER2> ...in the next 30 years</comment>


Show HN: My new website about hip hop - <AUTHOR> http://franklinsweekly.com/ <comment><AUTHOR> Hello!

I've created a website that links for hip hop tracks every Friday. It's like a blog but with just four quality links.

I haven't coded in a while so decided to update my coding habits by building this. It's built on Foundation 6.

There's a lot of hip hop being published every day. Franklins will attempt to collect the four tracks of highest quality every week. It's not made to make money, just a place for people to discover four quality tracks every Friday.

It's an attempt to curate a list of a more personal touch of music, like a sub genre in itself, by being selected by taste.

If you feel you got the time, please check it out! Cheers!</comment>


Most Uncured Bacon Is a Lie - <AUTHOR> https://thetakeout.com/uncured-bacon-label-nitrates-nitrites-celery-1834215295 <comment><USER2> The uncured Bacon producers aren't using HACCP approved processes (to the best of my knowledge) Botulism is only killed at very elevated salt concentrations, which may exist on the surface of a salt/sugar cured pork belly, but nowhere high enough on the interior. Any process that uses mechanical injection of brine could drive those spores into the meat interior where it's not subject to the required salt concentrations. Hanging hooks could drive botulism spores into the interior. The data on salt concentrations required to kill botulism spores is weak, so it's unlikely any which are on the interior will be killed. Then during cold smoke temps, botulism spores will burst forth and begin multiplying rapidly, without the required salt concentration to control them. Luckily, properly fried bacon is cooked at a temperature high enough to kill botulism toxin. Unfortunately, not all bacon is cooked in the manner. A James Beard award winning restaurant uses uncured bacon in their ramen broth base (I'm assuming it's prepared at temp/time which aren't sufficient to deactivate the botulism toxin). It's delicious, but I don't understand how the bacon could be processed to eliminate all risk. I happily eat it because nobody has yet fallen ill, but I haven't found the science which supports this method of processing.

</comment><comment><USER3> Isn't pork belly essentially uncured bacon (albeit not sliced)?</comment>


How do you evaluate yourself? - <AUTHOR>

how do you evaluate yourself ?how to know if I am getting better ? and if I am getting better how fast I am getting better ? <comment><USER2> This ones tough. In my life I do 2 things:

1) Logging. Measure what you want to improve, and then actually track the measurements. Also: log interesting things - your git repo tells all kinds of stories, if you want to put the effort in - because it's tracking something through time. Other forms of logging can also be used retrospectively when you didn't have a goal at the time of logging.

2) Ask for outside opinions (and log them). These are both better and worse than anything you can do yourself. Others have a much better perspective than you on the issues, but there is rarely an incentive to be honest in "friendly reviews".

These are both pretty obvious. And not silver bullets, but I do think they work particularly over longer time periods.</comment>


Ranking and Mapping scientific Knowledge - <AUTHOR> http://eigenfactor.org/map/

<comment><USER2> That's a nice little app to see the relationships between journals, but I was expecting to see a Google-maps mashup where they ripped out the locations from published papers then plotted areas of high intellectual concentration (pardon the pun), to show which areas of the world are focusing on certain topics.

Also, for the rankings, they could rank fields based on how often papers are cited by other fields. For example, if a certain Mathematics paper is frequently quoted by physicists, it would contribute to Mathematics having a higher rating than Physics (over a large number of papers, a better average rating would emerge).

</comment><comment><USER3> The journal rankings for physics appear to be bogus.

1\. Rev Mod Phys 2\. Mat Sci Eng R 3\. Annu Rev Phys Chem 4\. Phys Rep 5\. Rep Prog Phys 6\. Nano Lett 7\. Prog Polym Sci 8\. Adv Mater 9\. Phys Rev Lett 10\. Adv Polym Sci

Most agree that the top journals are: 1\. Nature 2\. Science 3\. Nature Physics 4\. Phys. Rev. Lett. 5\. ...

rev. mod. phys is a journal of review articles

<comment><USER4> "Most agree": my experience is that many physicists have different rankings than you suggest.

<comment><USER3> please share your experience! I don't take this ranking wankery too seriously. There are some papers that I really love that were published is lesser known journals. I guess I was talking about AMO and cond.mat. which are the majority of physicists. These lists are obviously biased towards experimentalists, although theory occasionally makes it into science / nature.

</comment></comment></comment><comment><USER5> Compare and contast:

<http://news.ycombinator.com/item?id=843709>

<http://news.ycombinator.com/item?id=396550>

These are different, but clearly it's the same sort of idea with different presentations or representations.

</comment><comment><USER6> There could be more subfields in here. I mean, one bubble for all of Mathematics?

</comment><comment><USER7> This is interesting but it is not ranking or mapping scientific knowledge. What is mapped is academic fields. The scientific content is not mapped or ranked.</comment>


Mistakes Job Hunters Make Online - <AUTHOR> http://online.wsj.com/article/SB10001424052748703389004575033583145567138.html

<comment><USER2> Basically, a list of recruiters to avoid :)

</comment><comment><USER3> There is a remedy against these vampires-recruiters: mainly: Contact the HR departments of companies directly, apply the resume on the websites and such.

<comment><USER4> Much better is to find the hiring manager and start a conversation with them.</comment></comment>


Yelp launches their daily deals in Seattle 1/27/11 - <AUTHOR> http://www.yelp.com/deals/panevino-seattle?bert=UDrLQLHkoYCvZc6dhWinLg:D

<comment><USER2> Yelp launched this months ago. See <http://techcrunch.com/2010/08/26/yelp- local-deals/>

<comment><AUTHOR> Ohhh I must have missed that TC article. Looks like they just launched in Seattle as of midnight. I updated the title.

Still interesting they're expanding. I wonder how they're competing with Groupon, LivingSocial, and Tippr..</comment></comment>


Show HN: Identibyte – Identify disposable and free emails in your forms - <AUTHOR> https://identibyte.com <comment><USER2> Hey all! We created Identibyte because it seems like every time we build a new product, we have to add a little code to make sure we're handling disposable email address accounts.

I love disposable emails, and I think it's great people use them to try out a service. We want to help companies (and anyone running a site with a signup form really), manage these better.

A lot of times users signing up with disposable emails don't need the time and attention that other users do - they just want to try out the product with little interruption. We encourage that!

Identibyte aims to helps the companies manage these accounts better. By pointing out information like this, you can focus your resources on the users who WANT and NEED it - instead of spreading everything evenly.

We've implemented this in our products by deleting disposable accounts after a certain time of inactivity. This helps us, and doesn't block people who just want to try out a product! Users who like the product almost always change their email to a real one.

I'd love to hear if this is something you would do, as opposed to completely blocking these signups?</comment>