Stan: a probabilistic programming language - <AUTHOR> http://mc-stan.org <comment><USER2> Stan is best viewed in my mind as a successor to BUGS (Bayesian Inference Using Gibbs Sampling) which more people may have heard of. In fact, there are lots of players in the probabilistic programming space now, personally I like the model of "Infer.NET" [1] from Microsoft Research, as I find variational and approximate variational inference a good solution to my problems and I like coding models in nearly C# more than in Stan format.

If you want to get started as fast as possible with computational Bayesian inference and don't need the performance and advanced features Stan, I'd recommend emcee [2] which is a lightweight python MCMC sampler.

Final recommendation to the HN crowd: Bayesian Data Analysis 3rd Edition [3] should be on your desk if you are thinking about using any of this software.

[1] [http://research.microsoft.com/en- us/um/cambridge/projects/in...](http://research.microsoft.com/en- us/um/cambridge/projects/infernet/)]

[2] [http://dan.iel.fm/emcee/current/](http://dan.iel.fm/emcee/current/)

[3] [https://www.crcpress.com/Bayesian-Data-Analysis-Third- Editio...](https://www.crcpress.com/Bayesian-Data-Analysis-Third- Edition/Gelman-Carlin-Stern-Dunson-Vehtari-Rubin/9781439840955)

<comment><USER3> Hi, stan dev here. I think viewing Stan as a better BUGS is helpful but limiting.

The syntax is similar, but the class of models Stan fits is far more general. The class of algorithms we have available also goes beyond MCMC, e.g., variational inference, optimization, and interfaces to Stan exist on all primary programming languages. It's more helpful to think of Stan as its own probabilistic programming language, and arguably the biggest entity with the largest user base.

<comment><USER4> >the class of models Stan fits is far more general.

This is mostly true, but last time I checked, Stan still couldn't sample discrete variables like BUGS can. Stan can only fit models with discrete parameters (e.g. finite mixtures) if the programmer is smart enough to integrate them out.

</comment></comment></comment><comment> dj-wonk One of my first questions was "What are the motivations for a language over a library?" After reading more, I see that Stan has both. There are integrations with R, Python, Julia, C++, Stata, the command line, and more.

To ask an intentionally over-simplified question: if the key value add of Stan in the functionality and approach -- why bother with the overhead of an external DSL? Why not use an internal DSL -- e.g. borrow data structures and/or syntax of another language?

Arguments for an external DSL are, hopefully: cross-language portability and domain-clarity for people.

But how does this play out if you want to support multiple languages and interop?

Let's say I'm using Python, for example, and want to build up a model programmatically. The example at [https://github.com/stan- dev/pystan](https://github.com/stan-dev/pystan) shows using a heredoc. That's simple and easy to start.

However, a big downside of a plain-text DSL is that it can be hard to manipulate programatically. To do that, each language implementation has to be 'business' of converting to and from text. This reminds me of the 'SQL' problem ... many language SQL wrappers spend considerable time munging text. Ug.

Given my bias towards data-oriented programming, I see another approach: why not use a rich data structure language (other than plain text) as the foundation?

Why not edn, for example? If the LISPy nature of that is not preferred, then why not something else? (Does the C-community have some kind of generic data description language? I wouldn't recommend JSON necessarily but that would be better than a hand-rolled DSL, in my opinion.)

Here's why I recommend starting with 'rich' data as opposed to parsing text...

I'm not saying human DSLs are bad. I'm just saying they don't _have_ to be the foundation. If you want a human external DSL, great -- but why not convert the human DSL to a canonical data structure? Then build everything around the data, rather than build everything around the text format?

<comment><USER5> As you mentioned, there are pros and cons to inventing the Stan DSL. Originally, we wanted a language that was not too different from the BUGS language because the BUGS family was what most applied Bayesians were using (if they weren't using some specialized MCMC algorithm for their problem).

And programmatic manipulation (while not impossible) of a Stan program was not high on our priority list. Since the underlying language is C++, you can bypass the Stan language and write (or generate) the C++ directly with a lot of effort. It uses standard (including Eigen) data structures.

<comment> dj-wonk > you can bypass the Stan language and write (or generate) the C++ directly > with a lot of effort

Did you mean "without" a lot of effort? :)

<comment><USER5> Unfortunately no, but a few brave souls have done so. It is slowly getting easier.

</comment></comment></comment></comment><comment><USER6> Stan is a fantastic language/library/tool. (Disclosure, I've taken 2 classes with Andrew Gelman[1].

For those who haven't used it, the typical usage is to use Stan in combination with other languages (most commonly R). In the Stan language you define a model, specifying the data you'll get, potentially transformations to that data, then a set of parameters you want to fit, then finally a model saying how those parameters interact with each other and the data. You can also define priors for parameters. Then typically you save that model, and using R pass in the variables to a Stan call. The resulting fit object is returned to your original environment automatically.

It actually fits the work style reasonably well. All data munging happens as before, but instead of having some complicated model expression in, say, a glm() call, you have it in Stan language.

If you're interested in this, Gelman has two great books. Gelman & Hill's Hierarchical Models[2] which is applied and geared towards social science researchers, and Bayesian Data Analysis[3].

[1] [http://andrewgelman.com/](http://andrewgelman.com/)

[2] [http://www.stat.columbia.edu/~gelman/arm/](http://www.stat.columbia.edu/~gelman/arm/)

[3] [http://www.stat.columbia.edu/~gelman/book/](http://www.stat.columbia.edu/~gelman/book/)

<comment><USER5> Also, the rstanarm[1] R package (disclaimer: that I co-wrote) will be released this month, which does not require the user to write any code in the Stan language. Instead, you specify the likelihood of the data (for a few popular regression-type models) using conventional R syntax and utilize Stan's algorithms and optional priors on the parameters to draw from the posterior distribution. In the demos/ directory of [1], we have replicated most of the first half of Gelman & Hill's textbook and are starting on the second half, which heavily utilizes our stan_glmer() function that is compatible with the syntax of the glmer() function in the lme4 R package.

[1] [https://github.com/stan-dev/rstanarm/](https://github.com/stan- dev/rstanarm/)

<comment><USER6> oh neat!

</comment></comment></comment><comment><USER7> Here is a well deserved plug for Cam Davidson Pilon's excellent book called Bayesian Methods for Hackers: [https://github.com/CamDavidsonPilon/Probabilistic- Programmin...](https://github.com/CamDavidsonPilon/Probabilistic-Programming- and-Bayesian-Methods-for-Hackers)

</comment><comment><USER8> Stan is absolutely a world class general Bayesian sampling tool. It replaces things like BUGS and uses fantastic algorithms for fast convergence. One of the major people behind it, Andrew Gelman, is a leader in applied hierarchical modeling and uses it to great effect in social sciences.

If you have even vague interest in applied statistics in fields with lesser ability to design experimental controls then you should investigate this tool.

</comment><comment><USER9> I found this recent paper showing how to fit and diagnose a simple non-linear model in Stan helpful. It is written from a statistician's perspective. [http://www.stat.columbia.edu/~gelman/research/published/stan...](http://www.stat.columbia.edu/~gelman/research/published/stan_jebs_2.pdf)

This one has more of a CS feel to it: [http://www.stat.columbia.edu/~gelman/research/published/stan...](http://www.stat.columbia.edu/~gelman/research/published/stan- paper-revision-feb2015.pdf)

Lastly, here is my own turbulent experience with getting started with Stan: [http://ericnovik.github.io/2015/08/14/Getting-Started- with-S...](http://ericnovik.github.io/2015/08/14/Getting-Started-with- Stan.html)

</comment><comment><USER10> The great thing about Stan is it is geared toward practitioners doing real work in science and social science, but still manages to push the boundary of stats research (NUTS, ADVI, penalized ML). Other probabilistic programming languages are more expressive, but are typically much harder to use for day- to-day work.

That being said, there's still a huge gulf between practitioners in science and social science that are building parametric Bayesian models and practitioners in the deep learning / ML community that are focused on building more general, scalable, machine learning algorithms for tasks like machine translation and question answering. It would be amazing if somebody could reconcile these communities by showing deep learning models can be expressed and fit effectively side-by-side with more parametric models.

</comment><comment><USER11> Does anyone have the experience with speed of convergence for some advanced models? And, defining sequence models in these probabilistic programming languages?

Seems most (if not all) probabilistic programming languages use sampling as a general algorithm for training models.

So, I guess it would be a hard job to express something like HMMs or CRFs that would fit into the framework (I guess one can easily train HMMs/CRFs with a fixed length of a sequence, but not undefined length).

<comment><USER12> You can easily encode any generative sequence model in e.g. Church (Scheme- based), Anglican (Clojure-based) or PyMC (Python-based). These are Turing- complete, so they can model any sampleable (computable) distribution.

See [http://forestdb.org/](http://forestdb.org/) for some examples. E.g. an infinite HMM.

I have no experience with Stan. There was some controversy on whether it is Turing-complete, but I cannot elaborate on that.

The obvious disadvantage of using a language that is too expressive for something as simple as an HMM is that you loose the nice performance guarantees given by all specialized algorithms such as Viterbi. Theoretically, they can achieve good efficiency by performing program transformations (e.g. with abstract interpretation). But in practice, we are still a bit far from that.

A nice trick is to implement your generative model in something very efficient (e.g. Probabilisitic C or something GPU-based). You can then forget the burden of having to encode your own sampling procedure, but at the same time inference might be tractable.

<comment><USER13> There's some research on automatically deriving efficient algorithms for classes of distributions, e.g. getting a specialized EM-like algorithm for anything where EM-like algorithms are applicable: [http://machinelearning.wustl.edu/mlpapers/paper_files/AA24.p...](http://machinelearning.wustl.edu/mlpapers/paper_files/AA24.pdf)

But obviously that doesn't cover classes as big as "anything expressible in Church", so there's a pretty big gap. It's possible that languages like Church could use techniques like that as part of an optimizing compiler that recognizes specializable patterns, the way regular compilers do loop transformations and auto-vectorization and such. But, like auto-vectorization, not that easy to do.

<comment><USER12> Sure, I mentioned abstract interpretation, but as you say all optimizing compiler techniques might be applicable.

</comment></comment></comment></comment><comment><USER14> hi. i'm one of the Stan devs. (i work on variational inference: ADVI).

happy to answer any questions here.

<comment><USER6> Yay Stan! So what's the deal with BBVI? I heard it's integrated in the newest Stan, but can I use it? What for?

<comment><USER14> glad to see the enthusiasm!

ADVI [1] is a variant of BBVI [2] where we fully leverage all of the amazing things that Stan has to offer (like automatic differentiation and automatic transformations of constrained parameters).

you can use ADVI to get an approximation to the Bayesian posterior. the advantage of using ADVI over sampling is that ADVI is typically faster for large models (both in terms of # of parameters and # of data observations). ADVI also a bit better at handling models with multi-modal posteriors, such as mixture models.

ADVI is currently in cmdStan (but not in RStan or PyStan). we're continuing to make the algorithm more robust.

[1] [http://arxiv.org/abs/1506.03431](http://arxiv.org/abs/1506.03431) [2] [http://www.jmlr.org/proceedings/papers/v33/ranganath14.pdf](http://www.jmlr.org/proceedings/papers/v33/ranganath14.pdf)

</comment></comment></comment><comment><USER15> Can anyone explain to me briefly what Stan actually does that's different from what I could do in Python without it? I don't want to have to read the entire research paper to get a description in layman's terms.

<comment><USER5> The number one thing in my opinion is that Stan's algorithm(s) for drawing from a posterior distribution produce samples that have much less dependence among adjacent draws than other simpler MCMC algorithms like Metropolis- Hastings or Gibbs samplers. Since it is often difficult to answer the question "How much dependence is too much in practice?", it is prudent to use the algorithm that yields the least dependence because the effective sample size (from the posterior distribution) per unit of wall time will usually be greater. PyMC3 has started to incorporate some of Stan's algorithms, although their implementations are not as far along.

<comment><USER16> I know some of those words...

<comment><USER17> Forget about the statistics and MCMC and all that. Imagine you have a function that is a black box and returns a floating point number. You can't see the implementation, but you want to find its maximum. How do you do it?

If you're like me you'd probably start with some kind of "grid search" by giving it evenly spaced parameters. Then you would evaluate it at each one and do some kind of gradient descent / ascent type algorithm. This will work, but in a high dimensional space (ie. a function with many arguments) you have no chance of covering even a tiny fraction of the parameter space. The key is, you don't want to waste time evaluating a set of parameters if it doesn't tell you anything new. ie. if you get back a similar answer to what you already had. This is what the GP means (I think) by trying to avoid "dependence between adjacent draws".

I could be wrong about all this, I'm trying to learn it too.

<comment><USER16> Thanks, that clarifies it a bit.

</comment></comment></comment></comment></comment><comment><USER18> It might give the right answers.</comment>


Blogging Fogbeam: Episode 2: The Journey of a Thousand Miles Begins... - <AUTHOR> http://fogbeam.blogspot.com/2011/06/blogging-fogbeam-episode-2-journey-of.html

<comment><USER2> Sponsoring splatspace/trifunc - do you think your target audience is there?

re: FFC - you mentioned "get out of the building" - has your experience been that physical "face to face" is inherently better than email/irc/im/phone?

Would be curious to know what "Friendly First Contacts" I could/should be making with virusade.com :)

<comment><AUTHOR> _Sponsoring splatspace/trifunc - do you think your target audience is there?_

"Target market" as far as customers? Nope, not really. That bit truly is about giving back to the community and being true to our values. Yeah, we get a link from <http://www.trifunc.org> and a link from <http://www.splatspace.org>, which maybe helps our pagerank or something... but it's really not about that.

That said, there is some value, I believe, in keeping (or building) "geek cred." When the day comes that we can start hiring employees, etc., I think it'll be valuable for people to see Fogbeam Labs as a company that is a genuine member of the hacking community and not just a money-grubbing, $EVILCORP.

 _re: FFC - you mentioned "get out of the building" - has your experience been that physical "face to face" is inherently better than email/irc/im/phone?_

Yes. So far, of the 15 or so Customer Discovery interviews I've done, all but 2 have been face to face, in person. The other two were by telephone. Telephone is better than nothing, but there's just something about sitting down together with somebody (preferably over lunch or drinks or something) where you can draw pictures, let the other person doodle on the picture, and just have that high-bandwidth, back and forth that includes body language, voice tone, facial expression, etc.

That said, keep in mind that I'm pretty old-fashioned in my beliefs about "doing business." Technology aside, I will still always prefer face to face communication, a firm handshake, eye contact, and all that "old skool" stuff __when possible __. No, I'm not enough of a luddite not to acknowledge the value of technological tooling for remote communication... but nothing beats face to face in my book.

 _Would be curious to know what "Friendly First Contacts" I could/should be making with virusade.com :)_

I think you're doing the right things so far. I signed up for your beta already. I'll email you when I have some thoughts. Better yet, let's get together for a beer again sometime soon. :-)

<comment><USER2> Thanks - your meetup today - probably a bit too far for me today - brier creek area would be much more conducive (that's only a 40 minute drive!)

<comment><AUTHOR> Understood. The meetup today is really kind of a one-off... I often go to that Barnes & Noble to work anyway, so I decided to just make today a small, casual H&F meeting. But, point taken, and I'll see about trying to do something more on the Raleigh side of the Triangle again, sometime soon. Durham really does seem to be emerging as the "hub" for startup activity in the area though, so there's a lot of allure to doing events there. :-)</comment></comment></comment></comment>


Do the dead outnumber the living? - <AUTHOR> http://www.bbc.co.uk/news/magazine-16870579

<comment><USER2> I always find this statistic about the number of people who ever lived to be incredible. It means that 6.5% of everyone who ever lived is alive right now. For me, that fact helps explain the incredible pace of change observable in every facet of civilization. There is an awesome amount of our humanity's intelligence on display at this very moment.

Then, when you consider how a lot of the 6.5% live (i.e. with extreme poverty and hunger), we are not even remotely reaching our collective capacity. The pace is going to get even faster as we achieve the Millenium Development Goals.

<comment><USER3> What really blows my mind about it has always been that there are properly more people alive today who can read Sumerian Cuneiform than there have ever been -- including back when there was something called Sumeria and they were using Cuneiform.

<comment><USER4> OT grammar question: I've noticed some comments from various posters use "properly" where I would have used "probably." Is this an alternate use of the word "properly" of which I have heretofore been unaware?

<comment><USER5> I believe grand-parent was using the term "properly" like it is used in math, i.e. "strictly greater than".

</comment></comment></comment></comment><comment><USER6> > _Do the dead outnumber the living?_

Yes, by a factor of 15 or so (from the article).

</comment><comment><USER7> There is considerable debate over when the first Homo sapiens sapiens appeared, but the consensus is that it was much longer ago than 50 000 years. 200 000 would be considered a fairly conservative number these days. This obviously skews the math a little.

<comment><USER2> 200,000 years ago is the approximate consensus for the arrival of Homo sapien; however the commonly held belief is that behavioral modernity occurred approximately 50,000 years ago.

Most of that doesn't really end up mattering because the last 10,000 years (marked at the origin of agriculture, 8000 BCE in the BBC table) have contributed disproportionately to the 107B figure. An extra 150,000 years at a baseline 1M population wouldn't throw the figure off more than 10-15%.

</comment></comment><comment> alok-g They used to tell children that people turn into stars when they die. Isn't it interesting then that the number of dead people is of the same order as the number of stars in a galaxy! :-)

</comment><comment><USER8> "Out of all the people that ever were, almost all of them are dead. There are _way_ more dead people. ... You're just dead people that didn't die yet."

It's a bit from a Louis CK routine, and the numbers appear to back him up.

</comment><comment><USER9> Given the average lifespan of a human on earth today (around 67), and the average population growth rate (somewhere between 1 and 2%), it causes an acceleration in the number of humans living and being born.

The notion that this growth is unsustainable is scare mongering. 1.5% growth per year is sustainable indefinitely to colonize the galaxy, if we can just keep the resources needed to support that growth flowing. At some point we will need to create a Dyson sphere around the sun to capture most of its energy, and harvest the raw materials in the asteroid belt between mars and Jupiter. As for living space, it will be superstructures orbiting the sun in the habitable zone.

<comment><USER10> >The notion that this growth is unsustainable is scare mongering.

Exponential growth makes this statement as wrong as it can be. It may be scaremongering to say it is unsustainable in the short term, but eventually that growth will have to stop, either due to voluntary decision-making on the part of humans everywhere, or due to a scarcity of resources. Or in fact both, given that the scarcity of resources will cause people to decide on reproducing less.

Growth in general is simply not sustainable in the long term. Isaac Asimov once estimated that at "today's" rate of growth (circa 1970), Earth would become a ball of human flesh expanding at the speed of light by 5000AD. Of course its a ridiculous prediction, but it illustrates that between now and then, something must change.

Look at this chart: [http://en.wikibooks.org/wiki/File:2nd_Order_Damping_Ratios.s...](http://en.wikibooks.org/wiki/File:2nd_Order_Damping_Ratios.svg)

Assuming 1 here refers to the maximum "sustainable" population, we'll either oscillate around it, overshoot and come back, or asymptotically approach zero percent growth. There is no such thing as a system with no feedback, which is allowed to grow without bound. It may seem like there are no constraints currently, but they're out there, and eventually we'll hit them.

<comment> a-priori The world population is not experiencing exponential growth. It's experiencing logistical growth (i.e, following a logistical function rather than an exponential function). We've just been seeing the first half of the curve, which does approximate exponential growth.

But we're starting to see it turn around. In the most developed countries, fertility has dropped to replacement or lower. As the remaining countries develop, their fertility rates are also dropping. It's reasonable to extrapolate from this that at some point in the future, the global fertility rate will be at or below replacement.

The constraints are microeconomic. It's expensive to raise children, especially when you factor in opportunity costs (this part is key to why world population won't stabilize at starvation levels). When you give people birth control, people seem to make a rational economic decision about how many children to have. That provides the feedback mechanism you mentioned.

<comment><USER7> The developing world has birth control but, in large parts of it (e.g. most of africa), most people aren't using it. This is for economic reasons. In these economies children are your pension fund. There's no RRSP funds or old age security payments. Just children who (hopefully) look after you in your old age. If you have only one or two you will either be an unreasonable burden upon them or simply be abandoned to starve.

The economic support system for seniors in the developed world is currently under a lot of strain too. In countries where pension funds are funded from current taxes the baby-boomer retirement is going to be a tremendous drain on society. The baby-boomer generation _should_ have tucked away a nice huge fund to pay them in retirement as well as support the greater demand on the medical system. Instead, they racked up huge amounts of public debt. Massive amounts of immigration and raising the retirement age have been the stop-gap measure of choice so far, but the developed world's support for senior citizens could easily revert to that of the developing world's without much warning!

So, yes, children are expensive, but for most of history and in large parts of the world today they are a necessary investment for retirement. In only the developed world of the past century have financial pressures favored fewer children, and that could easily change.

<comment><USER11> fertility rates are dropping worldwide. At some point in the next 30 years, it is expected to fall below replacement level.

Check out this graph comparing fertility rates. China and Iran (!) now have fertility rates below that of the U.S.

[http://www.google.com/publicdata/explore?ds=d5bncppjof8f9_&#...

<comment><USER12> Curiously, rates for Italy and Germany are separated from other developed countries by about .5% [http://www.google.com/publicdata/explore?ds=d5bncppjof8f9_&#...

</comment></comment></comment></comment></comment></comment><comment><USER13> 1\. read article

2\. * dead baby joke *

3\. ...

4\. Profit?</comment>


VeriFone Punches Square, Launches iPhone-based Card Reader - <AUTHOR> http://gigaom.com/2009/12/08/verifone-punches-square-launches-iphone-based-card-reader/

<comment><USER2> Details: <http://www.paywaremobile.com/>

Most likely requires a merchant account, while the whole point of Square is that it doesn't.</comment>


Why Google Became A Carrier-Humping, Net Neutrality Surrender Monkey - <AUTHOR> http://www.wired.com/epicenter/2010/08/why-google-became-a-carrier-humping-net-neutrality-surrender-monkey/ <comment><USER2> Google gave the market the option to go for a completely open environment with the Nexus One, and in the words of this wired article what did it get in return?

"Not cool enough."

The phone is brilliant, but the market went elsewhere with it's carrier locked subsidised junk models. That's just a market reality, like it or not. Google tried to "do the right thing" and the market went somewhere else. Making all these points about what they could've done instead which basically amounted to "become a direct competitor with Apple on the retail level" are not realistic considering everything we know about Google and it's views and practices on direct end user support.

If the market wants to shoot itself in the foot by choosing shitty products, that's their cross to bear. It's not reasonable to place the blame on one of the few companies that provided an out and had it thrown squarely back in their face. The only problem I fear is that due to the failure of the Nexus One they won't invest in a Nexus Two and the Android ecosystem will become a swirling morass of telco crippled product, thus ending differentiation between it and the competing iOS ecosystem.

The article does however make an interesting point, HP might do better with webOS, they are accustomed to end user hand holding and playing the retail / marketing game. If they can push a truly open ecosystem and manage to be successful in units moved as well, they may well end up being what Android might have been if the market had let it go in the direction Google had clearly wanted it to go from inception.

<comment><USER3> Not cool enough was hardly the problem - count instead:

1 - the phone came with waaaay substandard service - no live customer service, barely service via email

2 - no chance for the wide consumer market to play with the phone or interact with the phone with online sales only

3 - mixed reviews from the professional tech review community

So while I agree google made an attempt it felt to me much more like a half assed try in the context of this thread

<comment><USER2> Points 1 & 2 are what I was referring to when I pointed out that google just doesn't work that way, they're not going to open a plexiglass cube in the local megamall and handhold largely clueless end users via their version of the genius bar, it's simply not in their DNA. I acknowledge that that is in fact apparently an essential part of the path to market success in this arena as evidenced by the sales numbers, but I'm personally underwhelmed by it and simultaneously completely grok why Google has no interest in doing this.

Point 3 is news to me, every review I read on the N1 prior to purchasing one was a "this beats the snot out of the iPhone" level heaping of glowing praise, and now having had one myself for two months I see why; it's completely true. It's the Ubuntu story in a phone form factor all over again; having the clearly superior product does not mean that you end up with significant market share or business success.

<comment><USER4> _handhold largely clueless end users via their version of the genius bar,_

I think it would be great if google created a "genius bar". I'd call it the "let me just google that for you" bar.

Basically, you go to the bar and ask the bartender a question. He types the question into google and then reads you the answer. He might even refer you to the genius bar in the unlikely even that a quick google search doesn't solve your problem.

<comment><USER5> That is the square box on the right: <http://xkcd.com/627/>

</comment></comment></comment></comment></comment><comment><USER6> We didn't need no net neutrality in Canada.

We have two network providers and you have a free choice to get your phone, cable and wireless from whichever one of them operates in your town.

Sure we don't get 3G on our kindles and we got the iPhone a year after uzbekistan and we pay twice as much as you do for the data - but that's our choice as Canadian consumers.

<comment><USER7> Except we do get 3G on our Kindles, we got the iPhone years ago, and our 6 gig data plan is a better deal than AT&T's 2 gig top-end plan

<comment><USER8> > Except we do get 3G on our Kindles

For a $2 per book fee.

</comment></comment></comment><comment><USER9> FTA: ``Google could have fought. It had plenty of tools at its disposal. It could have made phones that worked on all of those networks, and then sued those companies if they didn't allow users to get fair plans.''

Does this sound at all like a well-thought-out, reasonable, or mature course of action to suggest that Google could have taken? I don't think so. What exactly does ``a fair plan'' mean? $0.50 less for unlimited texting? $10 less for wireless broadband?

FTA: ``That's fancy language for: Verizon and the nation's telecoms have yet again won, Google officially became a net neutrality surrender monkey, and you — as an American — have lost.''

Sounds to me like Wired is taking a shot at riding a wave of childish rage (and trying to get fat along the way).

<comment><USER10> A 15$ a month price reduction because they are not subsidizing a phone by ~400$?

<comment><USER11> T-Mobile does that; their unsubsidized (and contract-free!) plans are $20/month less. Of course neither Google nor T-Mobile ever bothered to tell anybody about that.

</comment></comment></comment><comment><USER12> The comments about phone exclusivity have nothing whatsoever to do with wireless net neutrality, but they do illustrate how everyone seems to want the wireless data and voice providers to turn themselves into valueless, interchangeable commodities fighting on margins much like major airlines. You really can't blame them for not wanting to play that game. Time and again other industries have shown that they will all lose. The exclusive phones and two year contracts are what's keeping their industry at a reasonable profit margin.

If I were running Verizon, AT&T, Sprint, etc, I probably would be pro net neutrality, as a lot of the consumers of wireless data will care about it and it will be a valuable feature, but I wouldn't be in a hurry to compete only on price and coverage either. If the people writing about this topic could only see past their loathing of the providers to be fair about it, and exclude off- topic rants about the providers trying to actually make a profit, they might further the discussion a lot better.

</comment><comment><USER13> Is anyone honestly surprised by this? Google played into the telecoms' hands, they created the first viable competitor to the iPhone and gave it to the telecoms saying "do whatever you want with this, feel free to modify it any way you want".

Google's vision of a market in which handsets are independent from carriers is absolutely poisonous to the industry's business model. Even though the iPhone is very successful I don't think verizon or sprint are interested at all in a world where they are just a "dumb pipe", they won't allow it.

Google lost this war because of their commitment to openness.

<comment><USER14> I'm not quite understanding the "gave it to telecoms" part - The telecoms don't make phones. Companies like Motorola, Samsung, and HTC do.

And while Google "gave" it to them, they've all dumped incredible amounts of engineering hours into Android and their derivatives -- any illusion that they just ran a Ubuntu install on their Galaxy S and that was it is utterly asinine around parts like this.

Seriously, the "giving it away" bit is dumb, and while it sells on the non- technical sites, it is an embarrassment on HN.

<comment><USER13> I find your lack of courtesy to be the embarrassment to HN.

I was simply try to the make the point that by pushing an open source OS (as in manufacturers could alter the OS to include anything the carriers insisted) Google blindsided itself. They expected to create something that would change the handset/carrier/customer relationship, instead they just enabled the next generation of the same thing. The Nexus One is evidence of this.

</comment></comment></comment><comment><USER15> So from moral prospective is it better to support Android or iOS at this point? Its a fine case of damned if you do damned if you don't.

<comment><USER16> True. IF I understand the implications of the deal between Google and Verizon correctly (and to be honest, there's a good chance I don't...), I would argue that what they're doing is far worse than Steve Jobs saying, "You're holding it wrong."

At least, the actions of Apple (and Jobs) is limited directly to developers and consumers who choose to use (or develop for) their products.

Verizon and Google? This affects all of us.

Edit: But to answer your question: Let's support the little guy and go webOS!

</comment></comment><comment><USER17> This article is absolutely ridiculous. Wired should be ashamed -- it is sensationalist, unrealistic, and absolutely dishonest in its characterization of the Google/Verizon policy proposal. The salient mistakes:

1\. "Google and Verizon announced Monday, as part of their bilateral net neutrality trade agreement they want Congress to ratify, that open wireless rules were unneccessary.

"We both recognize that wireless broadband is different from the traditional wire-line world, in part because the mobile marketplace is more competitive and changing rapidly," the joint statement said. "In recognition of the still- nascent nature of the wireless-broadband marketplace, under this proposal we would not now apply most of the [Net Neutrality] wire-line principles to wireless, except for the transparency requirement."

That's fancy language for: Verizon and the nation's telecoms have yet again won, Google officially became a net neutrality surrender monkey, and you — as an American — have lost."

The proposal[1] specifically notes that the wireless exemption is time- limited-- it is noted that _"at this time"_ these rules would not apply. The proposal includes an annual review of this position, and the transparency requirement attempts to ensure that this review could be conducted fairly and with good information.

2\. "Google could have fought. It had plenty of tools at its disposal. It could have made phones that worked on all of those networks, and then sued those companies if they didn't allow users to get fair plans."

Really? And destroyed any hope of Android ending up on those carriers in the future? Does anyone really think this is a sane proposition? (Does anyone believe that Google wouldn't be painted as a litigious bully by the very same critics throwing around such absurd language as "carrier-humping surrender monkeys"?)

3\. "The FTC would have had a reason to pry into unfair business practices. Google could have eschewed online-only selling and partnered with the many independently owned mobile phone shops around the country, so that potential customers could play with the device before plunking down $500."

The reasons for the Nexus One's failures are complex, but I certainly think that one of them was that _many consumers don't want to pay $500 for a device_ , and are more than happy to sign multi-year contracts in order to get a subsidy on a smartphone.

4\. "Google easily could have attached conditions to all Google-powered Android phones, banning carrier software that can't be removed just as easily as any other app. (Try getting rid of Sprint's Nascar app on the EVO — if you don't have root, it can't be done.). These conditions also could have banned the blocking of Android 2.2's built-in ability to be a Wi-Fi hot spot, which both Sprint and Verizon have crippled."

I know the "open" crowd isn't a fan of these management policies-- I'm not either-- but it seems hypocritical to assert that certain kinds of customizations shouldn't be allowed on open-source software. The author essentially wants Google to be the arbiter of what "openness" means, and moreover, to apply an unequal standard to customers versus carriers.

Boo-hoo, right? Verizon certainly can deal with getting the short end of _that_ stick. But the way they would most likely do that is to drop Android altogether. At which point no one gets to customize it at all.

I am continually mystified by the legions of Google critics who expect that, because they are huge, they can do whatever they want-- completely ignoring the fact that they are huge _because they often don't do what they want to_. Android is a success precisely because it balances openness with pragmatism. At times I disagree on the balance that Google has chosen, but I don't for a minute believe that Google can simply ignore reality and force carriers to accept a model of the internet that they fundamentally disagree with.

We can argue about the merits of particular Google decisions (like, say, the wireless exemption in the current policy proposal), but I don't think we get anywhere by mis-characterizing them and using the kind of polarizing, childish language like "carrier humping surrender monkeys".

[1]: [http://docs.google.com/viewer?url=http://www.google.com/goog...](http://docs.google.com/viewer?url=http://www.google.com/googleblogs/pdfs/verizon_google_legislative_framework_proposal_081010.pdf)

</comment><comment><USER18> The Nexus One launch and seemed ghetto and not something I would have expected from Google.

</comment><comment><USER19> "It could have made phones that worked on all of those networks, and then sued those companies if they didn't allow users to get fair plans."

the networks require different antennas. no phone can work on all of them and be competitive.

google got verizon to agree to neutrality on the wire and can continue to fight for fair treatment on wireless. i call this progress.

</comment><comment><USER20> Google spokeswoman: "We have taken a backseat to no one in our support for an open internet. We offered this proposal in the spirit of compromise. Others might have done it differently, but we think locking in key enforceable protections for consumers is progress and preferable to no protection."

I have to agree, Google is getting flak here although they are the only company that stood for net neutrality. They are being pragmatic here and everyone is ignoring the good points in this agreement. Also being ignored the fact that this is non-binding agreement that is designed to speed the debate.

<comment><USER8> This deal is just a fig leaf. Google supported net neutrality when it suited their business goals. The moment Android took off it became a non-issue for them. If anything, they should be taking more flak. Google's enjoyed an incredibly soft treatment by the press thanks to their positioning as a company that puts ethics before profits. Yet on the issues that matter, it's increasingly clear that Google's ethics are highly situational.

<comment><USER21> They succeeded in getting Verizon to agree to net neutrality on the wired internet. Isn't that a strict improvement over the status quo?

<comment><USER22> No it is not. The status quo is that no one has yet made any binding decisions, and the telecom operators will try to get away with what they can until the FCC slaps them down.

Google just succeeded in getting Verizon to agree to net neutrality on the wired internet in exchange for removing any possibility of net neutrality on the wireless internet (which Google claims is the future of internet).

<comment><USER21> >in exchange for removing any possibility of net neutrality on the wireless internet (which Google claims is the future of internet).

[citation needed] This is explicitly not true according to the text of the agreement. The agreement says that it is too early to determine whether net- neutrality provisions are necessary for wireless networks, because it's a newer market and there is much more competition.

<comment><USER23> That kind of "let's wait and see how it looks until it evolves before we regulate" attitude is precisely what caused the current stagnation in the wired market.

The market fundamentally is under heavy regulation; wireless can't work without heavy-handed regulation of who can use what spectrum. We can't just stop at that kind of draconian regulation (which is necessary) and say that a little thing like non-discriminatory access to your absolute monopoly is too much.

<comment><USER24> However there are fundamental economic reasons why the wired market should lead to natural monopolies and the wireless market should lead to more competition. Therefore there is good reason to wait.

Furthermore bandwidth is limited. The proposal is not what Google wants, it is what they were able to get Verizon to agree to. I read that clause as very much of an, "We agree to disagree, and agree that our areas of agreement are worth pursuing anyways."

</comment></comment></comment></comment></comment></comment></comment><comment><USER25> What? A big corporate behemoth like GOOG being a hypocrite? How utterly, utterly shocking.

</comment><comment><USER26> Unfortunately it seems no one cares about corporate ethics c.f. Facebook.

I guess my only option is to buy a few shares of GOOG.

</comment><comment><USER27> When did our culture shift to this point that everyone puts their problems on the government?

Everyone complains about Apple's closed iPhone ecosystem, then pesters the government to condone jailbreaking and unlocking, which go against the iPhone's terms of service.

Comcast is caught throttling bittorent, and instead of switching providers to satellite or Verizon or anything, everything goes to the government and demands regulation on net neutrality.

Since when did it become the american standard to complain to the government when you don't agree with the terms of service instead of just doing it the old fashioned way and speaking with your money? If high speed internet is so important to you and Comcast is the only carrier in your area, then you are at their mercy - they paid money to expand their service to your region and service you - you don't get to demand that they service you in the most favorable way.

Everyone wants to have their cake and eat it to, and appealing to legislation just seems wrong.

P.S. If you disagree with me and believe that legislation limiting the contacts that can be signed between two parties is necessary, please reply and explain why instead of downvoting an alternate point of view. This isn't reddit.

<comment><USER28> > If high speed internet is so important to you and Comcast is the only > carrier in your area

This is usually due to a local monopoly having been granted by local government. I think contracts between corporations and government that have the result of reducing competition in the marketplace should be legislated against.

Take a look at the competition that exists between ISPs in places like Europe or Australia to see how it could be. Capitalism works best when competition exists. Obviously it's in the best interest of corporations to attempt to reduce competition in their industry. That doesn't mean they should be free to do so.

<comment><USER29> "I think contracts between corporations and government that have the result of reducing competition in the marketplace should be legislated against."

Or just not legislated into existence in the first place, would have been a good start. But now we need more legislation to undo the damage of the monopoly granting legislation.

So it's not so simple as blaming it on just government or just corporations. It is the collusion of large corporations and government eliminating competition that is the real problem.

</comment></comment></comment><comment><USER30> It's a series of tubes. And if you don't understand, those tubes can be filled and if they are filled, when you put your message in, it gets in line and it's going to be delayed by anyone that puts into that tube enormous amounts of material, enormous amounts of material</comment>


Show HN: Remembered.io – Easily remember everything worth remembering - <AUTHOR> https://remembered.io/

<comment><AUTHOR> Hey HN, we built this recently because we saw a gap not being filled by calendar/note/todo apps. We noticed that even after writing down notes/ideas we wanted to remember, we'd forget them because our brains just aren't good at keeping things top of mind. It's possible to set manual reminders but it quickly becomes too much effort.

Would love to hear what people think (constructive feedback please). Happy to answer any questions too. Thanks!

<comment><USER2> I'm a huge fan of spaced repetition and a regular Anki user. I think the value proposition is a little too vague here. It seems like your target market is people that don't already know about or use spaced repetition. In order to win them over, I think you need a more specific reason to use the product.

I saw another YC company recently that is using spaced repetition as a means for corporate training. I'm not saying it's the best or correct market, but that strategy seems plausible to me.

Good luck!

<comment><AUTHOR> Thanks! That was helpful. I agree we can do a better job with the copy on our landing page.

I use Anki as well for learning Chinese, but the use case is different for me.

What YC company are you referring to?</comment></comment></comment>


To keep EpiPen sales up, Mylan threatened states, sued making bogus claims - <AUTHOR> https://arstechnica.com/science/2017/04/mylan-bullied-sued-states-to-get-them-to-use-epipens-court-documents-show/ <comment><USER2> Of note: the CEO of Mylan, Heather Bresch, is also the daughter of the senior senator representing West Virginia, Joe Manchin, who was also the governor of the state from 2005-2010. Joe's wife (and Heather's mother) Gayle Manchin took over as the head of the National Association of State Boards of Education in 2012, and helped push through the 'EpiPen Law' in 2013 that gave funding preference to schools that stocked them. [0]

Much like fish, the head tends to rot first.

0: [https://www.usatoday.com/story/news/politics/2016/09/20/fami...](https://www.usatoday.com/story/news/politics/2016/09/20/family- matters-epipens-had-help-getting-schools-manchin-bresch/90435218/)

<comment><USER3> It gets even better (Mylan is in my hometown), from Wikipedia [1]:

"In 2007..the Pittsburgh Post-Gazette reported that Bresch had claimed to have an MBA degree from West Virginia University, but the university disputed that. The university subsequently awarded her an EMBA despite her having completed only 26 of the required 48 credits. Her father was governor of the state of West Virginia at the time.

In the ensuing controversy, the university announced in April 2008 that it would rescind Bresch's degree. Michael Garrison, WVU President at the time, was reported to be "a family friend and former business associate of Bresch" and a former consultant and lobbyist for Mylan. After a faculty vote of no confidence, Garrison and several university officials subsequently resigned."

[1][https://en.m.wikipedia.org/wiki/Heather_Bresch](https://en.m.wikipedia.org/wiki/Heather_Bresch)

<comment><USER4> Sounds like a lot of home cookin' going on there, with the EpiPen being only the crescendo of scandals caused by this group of family & friends.

It would be great if we could clean house and replace those acting in bad faith.

<comment><USER5> What recourse will we have against them? Do any of the punishments for our ruling class have teeth?

If you win, you become richer. If you lose, you... Lose your job at the state capitol?

<comment><USER6> Worse still: if you lose other people's money you get bailed out (oh, and no bonus for like 3 years or so).

<comment><USER7> what? no.. you can't take away their bonuses. that would be unfair: [http://www.nytimes.com/2009/03/15/business/15AIG.html](http://www.nytimes.com/2009/03/15/business/15AIG.html)

</comment></comment></comment></comment></comment></comment><comment><USER8> Wow! Hospital supplies, prescription drugs and many other health care markets are effectively un-enterable by new innovative entrants through similar kinds of illegal, anti-competitive arrangements....probably one of the reasons we haven't seen a YC hospital supply startup... '

"EpiPen's sales and expanded markets brought in more than $1 billion in revenue for Mylan." "The company's CEO, Heather Bresch, is one of the highest- paid CEOs in the industry, earning nearly $19 million annually." "Mylan...had...illegal deals with schools to undercut competitors and allegedly scammed federal and state regulators out of millions..."

<comment><USER9> Actually, a good friend of mine just filed for a patent on a widget which will directly cut into Mylar's epipen sales. It's gonna be a tough startup, I'm sure, but he's doing it.

<comment><USER10> So what, 6 years to get through the FDA?

</comment></comment></comment><comment><USER11> But the companies have to keep making abnormal profits, because the US subsidizes R&D for the rest of the world!

<comment><USER12> Their main r&d seems to be to stop others from getting approval for cheaper competitors.

<comment><USER13> No one said R&D needs to stay in product world. It's "business practice" R&D that rakes in the big bucks!

<comment><USER14> We are no strangers to this business^H^H^H^H^H^H^ legal plan.

[https://davidkellogg.files.wordpress.com/2015/12/oracle- org-...](https://davidkellogg.files.wordpress.com/2015/12/oracle-org- chart-300x195.png?w=500)

</comment></comment></comment></comment><comment><USER15> Wow. There is noway such strategies are not sanctioned at the top level. It would be interesting to see the CEO charged with malicious prosecution (which is not limited to criminal justice proceedings but includes fraudulent tort claims and suchlike).

</comment><comment><USER16> It seems that it would uncontroversial and acceptable to both sides of Congress if they pursued creating a legal framework for making this sort of activity both illegal and with meaningful punishment. We can ban people from playing baseball for life, why not ban them from participating in the healthcare community?

<comment><USER15> I don't think so. Congress is full of corrupt people who would have too much to lose from passing such a law.

</comment></comment><comment><USER17> Why does an adrenaline shot have to cost that much?

<comment><USER18> Because most people don't know how to use hypodermic syringes. And because either too little or too much might kill the patient.

<comment><USER17> It doesn't explain or justify the price. It only explains the need for using such devices.

<comment><USER18> True, it doesn't account for Mylan's price. That mostly reflects anticompetitive behavior. But even the least expensive versions cost much more than multidose bottles and syringes.</comment></comment></comment></comment>


New York City photographed with the Game Boy Camera in 2000 - <AUTHOR> http://www.ironicsans.com/2014/05/new_york_city_in_2000_photogra.html

<comment><USER2> The linked colour pictures are also pretty neat.

[http://www.ironicsans.com/2007/09/idea_color_photos_with_the...](http://www.ironicsans.com/2007/09/idea_color_photos_with_the_gam.html)

<comment><USER3> Awww man...I've got a gameboy camera lying around as well, and I have that Mad Catz cable adapter he's talking about, but I could never get it to work :(

Also, his method reminds me of these old Russian photographs:

[http://petapixel.com/2014/03/10/incredible-color- photographs...](http://petapixel.com/2014/03/10/incredible-color-photographs- of-early-20th-century-russia/)

</comment></comment><comment><USER4> Neil young's Silver & Gold album cover was shot by Neil's daughter Amber with a Game Boy Camera.

[http://en.wikipedia.org/wiki/Silver_%26_Gold_(Neil_Young_alb...](http://en.wikipedia.org/wiki/Silver_%26_Gold_\(Neil_Young_album\))

<comment><USER5> Wow, that is actually really cool.

I remember seeing ad's for this camera when it came up, but never actually saw a picture from it until recently. I really love that weird tech like this existed.

</comment></comment><comment><USER6> Great demonstration of "toy level camera tech" 14 years ago!

The interesting thing about the sensor in these things (Mitsubishi M64282FP) is that it has an _analog_ output, so it's capable of much higher bit depth than in Nintendo's implementation of it. There are various other (mostly robot vision) projects using it that you can find on the Internet, and you'll see that the images are far less pixelated.

I also think it's rather amazing how much the tech has progressed - 15 years ago, the cost of a 128 x 123 (there's 5 black lines probably for BLC) - 16KP - monochrome array fixed-focus sensor module would be around the same as a 5MP AF colour module today, of the type used in smartphones.

<comment><USER7> Sounds interesting. Can you explain how a higher bit depth is used to produce images with less pixelation (higher resolution)?

<comment><USER8> It's not about resolution. Part of the reason those images look so pixelated is because they're quantized down to 2 bits per pixel: you only get white, light gray, dark gray, and black. Any intermediate shades have to be represented by dithering, which is what creates that stippled, checkered appearance.

If the images had the same resolution but higher bit depth, you wouldn't be able to make out much more detail, but the images would be a lot smoother and less blocky-looking.

EDIT: I used today's Wikipedia featured picture as an example.

Original: [http://en.wikipedia.org/wiki/File:Kumar_Anish.jpg](http://en.wikipedia.org/wiki/File:Kumar_Anish.jpg)

256x256, 8 bits per pixel: [http://i.imgur.com/YVIS2g3.png](http://i.imgur.com/YVIS2g3.png)

256x256, 2 bits per pixel: [http://i.imgur.com/iTFslZx.png](http://i.imgur.com/iTFslZx.png)

256x256, 2 bits per pixel, with dithering: [http://i.imgur.com/LnlmORI.png](http://i.imgur.com/LnlmORI.png)

<comment> robert-boehnke I imagine this is because the original Gameboy only had a 2 bit color palette.

<comment><USER9> Right. The Game Boy Color changed that, but it wasn't black-and-white.

</comment></comment></comment></comment></comment><comment><USER10> The subway speeding past photo is interesting- the door is clearly slanted to one side. Is this an artifact of the camera scanning line-by-line relatively slowly? Or something else?

<comment><USER11> [http://en.wikipedia.org/wiki/Rolling_shutter](http://en.wikipedia.org/wiki/Rolling_shutter)

Common on certain types of digital cameras today as well (especially smaller ones like mobile phones and P&S cameras).

</comment></comment><comment><USER12> Wow. I remember that cam.

Fun to do. Put one of those pics on your screen and walk back a few meters. For example the picture of the guy in the subway sleeping. You'll notice the details start appearing because your brain is filling in the details.

</comment><comment><USER13> A music video created with a gameboy camera: [http://vimeo.com/19868821](http://vimeo.com/19868821)

It's also worth noting that the gameboy camera also contained a very weird and limited sequencer called trippy-H, which you accessed by first completing an objective (take so many photos or make an animation or something).. and then shooting the right thing in the shooting game.

Here's a demo of it: [https://www.youtube.com/watch?v=7SDQ7vF5CIQ](https://www.youtube.com/watch?v=7SDQ7vF5CIQ)

I loved that the gameboy camera had a ton of completely hidden weirdness (pre- internet!) which you found out and shared with your friends.

</comment><comment><USER14> This reminds me of another cool toy camera, the PXL-2000 - which recorded digital video onto a compact audio cassette tape.

[http://en.wikipedia.org/wiki/PXL-2000](http://en.wikipedia.org/wiki/PXL-2000)

<comment><USER15> I went to a PXL-2000 film festival about 15 years ago in Vancouver which was absolutely brilliant. It's amazing how creative people get when forced to deal with constraints.

Speaking of constraints, the camera apparently had a problem with its RF filter, which resulted in a lot of static when you were recording to the audio tape. Someone figured out you could hack in your own RCA video out connector, so most of the films shot with the camera have been modded to get a better picture. Here's a link on how to mod one: [http://users.speakeasy.net/~joem/Pxl/guide.html](http://users.speakeasy.net/~joem/Pxl/guide.html)

</comment></comment><comment><USER16> Here's a web app that will "gameboyify" a photo

[http://www.tabletop-pixel.com/retroizer/](http://www.tabletop- pixel.com/retroizer/)

<comment><USER17> [http://i.imgur.com/0IAaSYp.png](http://i.imgur.com/0IAaSYp.png)

</comment></comment><comment><USER18> Quality doesnt seem to be that much different from the first ever digital camera, the cromemco cyclops ([http://en.wikipedia.org/wiki/Cromemco_CYCLOPS](http://en.wikipedia.org/wiki/Cromemco_CYCLOPS)). Judged by the few examples that can be found via Google images, that is.

Incidentally, that camera used rather interesting technology. Its sensor was a regular CMOS ram chip with the opaque cover replaced by a transparent one. Bits would flip back from 1 to 0 due to the light projected onto it.

</comment><comment><USER19> I had one of those back in the day. It wasn't just a camera, it also had some amazingly rudimentary photo editing capabilities, which were fun, as well as lots of amazingly weird games.

If you'll excuse the Cracked link, there are also some weird easter eggs: [http://www.cracked.com/article_19332_7-creepy-video-game- eas...](http://www.cracked.com/article_19332_7-creepy-video-game-easter-eggs- youll-wish-were-never-found_p2.html)

</comment><comment><USER20> I've been wondering for a long time why Nintendo never went ahead and made a smart phone. This really cool blog post sort of twisted the knife to me about their possible missed opportunity.

</comment><comment><USER21> Url changed from [http://www.designboom.com/art/game-boy-camera- photographs-05...](http://www.designboom.com/art/game-boy-camera- photographs-05-13-2014/), which pointed to something that pointed to this.

Submitters: HN prefers original sources. Please check what you post for cases where it's just pointing to something else.

<comment><USER22> The current URL is far more interesting with its links to his colour photographs.

<comment><USER23> I've been following his blog for a few years now. He's done some other really neat stuff like histogram work ([http://www.ironicsans.com/2007/09/idea_the_histogram_as_the_...](http://www.ironicsans.com/2007/09/idea_the_histogram_as_the_imag.html)) and sort of an inverse Tetris ([http://www.ironicsans.com/2009/01/i_wrote_it_you_made_it_mon...](http://www.ironicsans.com/2009/01/i_wrote_it_you_made_it_montris.html)). Then there are the inventor profiles ([http://www.ironicsans.com/interview/](http://www.ironicsans.com/interview/)).

<comment><USER24> A script for generating images with self-similar histograms (read the first article!) - [http://www.joshmillard.com/2007/10/04/retro-histo-making- an-...](http://www.joshmillard.com/2007/10/04/retro-histo-making-an-image-fit- your-histogram/)

<comment> jack-r-abbit The histogram work is awesome. I know of some people that would love this for crypto-puzzles.</comment></comment></comment></comment></comment>


Access to HN posts - <AUTHOR>

I am trying to use ec2 machines to scrape HN posts, but it isn't working.  Specifically, it seems like HN is rate limiting request from ec2 machines.  Does HN not want people scraping HN?  How can I get access to all HN posts? <comment><USER2> You can scrape HN as long as you respect the robots.txt[1] and don't retreive more than a couple of pages per minute.

Have you considered just pulling the data from HNSearch's API[2] or the one by iHackerNews[3]?

[1] <https://news.ycombinator.com/robots.txt>

[2] <https://www.hnsearch.com/api>

[3] <http://api.ihackernews.com/>

</comment><comment><USER3> _Does HN not want people scraping HN?_

I don't remember the details, but I think pg has expressed some desire to not have people scraping the site, or at least not to scrape often. I believe the justification was that too many bots crawling/scraping the site hurts performance for everybody else. You might try searching the old posts for more info on the topic.

 _How can I get access to all HN posts?_

Try <http://api.ihackernews.com/></comment>


The Retrade - <AUTHOR> http://www.feld.com/archives/2016/02/the-retrade.html <comment><USER2> Has anyone here had this happen to them? It is hard (and scary) to imagine a situation where I was dealt with in this way and was still OK with having the investor on my board.</comment>


The Geek-Boy Irony Behind Mark Zuckerberg's Tech Lobby - <AUTHOR> http://www.fastcolabs.com/3010423/the-geek-boy-irony-behind-mark-zuckerbergs-tech-lobby

<comment><USER2> "Geek learning is highly dependent on informal, problem-driven, and peer-to- peer social learning." Yes, but it wouldn't need to be if school curricula identified it as important. Kind of like learning about sex before social policy decided sex education might be a good idea.</comment>


Microsoft's Courier 'digital journal': exclusive pictures and details - <AUTHOR> http://www.engadget.com/2010/03/05/microsofts-courier-digital-journal-exclusive-pictures-and-de/ <comment><USER2> If the following are true: 1) The device renders are close to reality 2) The software videos are close to reality 3) Users are allowed to install any software they want 4) Programmers aren't subjected to a crapshoot approval process 5) The price is realistic for the masses 6) A sizeable ad campaign is launched to promote the device

I can really see this working.

<comment><USER3> You can also expect to win the lottery. A couple times. In a row.

</comment></comment><comment><USER2> Two more thoughts; IF it delivers:

1) This would be killer in education

2) Would artists love Apple as much when Microsoft is delivering products like this?

</comment><comment><USER4> This looks like the tablet I've always dreamed of. Something like a high tech notebook. I'm having second thoughts on picking up an iPad now.

<comment><USER3> > I'm having second thoughts on picking up an iPad now.

Seriously? Comparing one box that runs the same software iPhones and iPods run to another than runs... Windows Mobile?

<comment><USER4> It doesn't really matter which OS it's running if it does what I'm looking for. I know the iPad will have a wealth of apps that do what I want (or I'll write my own) it just won't have the pen input and the notebook feel I'm looking for.

I've had Windows Mobile devices before and been severely disappointed but that doesn't mean I'm not willing to give it another try, especially based on what I see in this video.

<comment><USER5> Hey if you want pen input you might look at getting a pogo sketch from tenonedesign.com - I've got one that I use on my mbp with inklet and I've tried it on my 1st gen ipod touch and it works there too. I'm guessing with its beefier processor the ipad will work even better.

</comment></comment></comment></comment><comment><USER6> While I frown upon Apple's policies sometimes, one thing I liked about them is when they announce something, they deliver it.

Courier has been publicized for sometime now but there's still no release date. Same with the slate they announced with HP during CES. They said later this year but no follow-up since.

<comment><USER7> Oh come now - Apple milked the hype machine for every last drop of publicity prior to the actual announcement, to the point of it being a news item _before_ it took place - they just kept the images and specs locked down.

I can't blame MS for testing the waters and getting preliminary feedback. I personally prefer this approach, since it helps them refine and helps me to budget attention and/or money for an interesting new technology.

<comment><USER6> I'm not saying Apple didn't ride the publicity machine but at least they are delivering what they promised.

Courier, any idea when it will be released? How about Slate? Remember Origami/UMPC? Yeah they released but they didn't focus much energy on it.

I'm not an Apple Fanboy. I own a Zune and a PC.

</comment></comment></comment><comment><USER8> Call my a cynic or a hater, but I won't believe a word of it until I see a working prototype that resembles the mockups in functionality. Until that happens it's speculation, artist renderings and vaporware.

</comment><comment><USER9> Looks awesome on video, but that isn't surprising. Who would make an ad which wouldn't make their product look awesome?

ps. @2:55 on the second video: IE6 just doesn't stay dead

</comment><comment><USER3> Apple's Newton is sooo doomed...

</comment><comment><USER10> The video on top seems newer than the one below (seen last fall?). I still keep debating whether it's going to be a good experience to deal with holding two screens at once, that along with what the hardware price might be at that point (1k+ for a tablet? Hey, tablet PC's sell for that much already)

Hopefully a vendor will pair this transreflective screen for a more natural drawing experience.

</comment><comment><USER11> The video's look impressive as hell. With a lot of work and usability testing, they could have a killer product.

Just imagine how natural and fluid your work could be - mocking up UIs, keeping track of work items, sketching class/module diagrams etc etc. This is the first thing I've seen that has the _potential_ to be better than notebook and I really do hope they realize that potential.

</comment><comment><USER12> Anyone else think it's strange that a demo of a Microsoft product is using Google Maps (instead of Bing Maps)?:

<http://www.viddler.com/explore/engadget/videos/1174/16>

</comment><comment><USER13> Yes, another pretty concept video, now what? I can't help being cynical. I mean, since when has MS actually delivered on their promises?

<comment><USER14> Windows 3.1 was the last time I can remember. Or maybe when Office originally came out?

</comment></comment><comment><USER15> Will it be possible to reverse the menu page's home and set it on the right- hand display... to accommodate lefthanded users?

<comment><USER16> We don't even know if it'll be possible to fully close the notebook yet.

<comment><USER17> From the article: it's under an inch thick, weighs a little over a pound, and isn't much bigger than a 5x7 photo when closed.</comment></comment></comment>


Working computer made out of Minecraft blocks - <AUTHOR> http://www.boingboing.net/2010/09/28/working-computer-mad.html <comment><USER3> The good: he spent a lot of time learning about logic circuits

The bad: he spent a lot of time making a simulation of a logic circuit that has no practical application I can imagine

It's probably a net plus overall, since learning and modeling what one knows is a great way to reinforce it, but one is left with the impression that this is a person who can spend a lot of time on things.

I hate to be a prude, and everybody loves a hobby, but we keep seeing these stories of guys who spend hundreds or thousands of hours on these very unusual and detailed configurations of virtual goods -- the maximized Sim City guy comes to mind. Something about our praising this behavior bugs me. I guess it's not clear to me the difference between unnaturally focusing in on small things - -like in Aspergers -- and just having a hobby. Aside from this guy, who seems wonderfully well-balanced, I wonder if we're not praising people who might actually need help.

I don't know. I probably didn't say that as well as I could.

<comment><USER4> I'd rate this a lot higher than the guy who made a big effort out of trying to build Simcity city with the largest possible population.

SimCity guy spent years on his city. I don't know how long this guy spent, but it can't have been nearly that long since the game is only a year or so old. It probably took some reasonable number of hours -- probably less than the average Minecraft player spends playing Minecraft _without_ producing anything that anyone ever wants to look at.

Secondly, SimCity guy was trying to get the highest possible score within the parameters of the game as defined, which is less creative than this guy who is using the game in an unexpected and novel way.

<comment><USER5> > _"this guy who is using the game in an unexpected and novel way."_

MineCraft Alpha was built with logic-circuit design in mind [1]. This guy built a pretty extreme example, but it's not any more "unexpected and novel" than, say, making a very large castle.

[1] <http://minecraftwiki.net/wiki/Redstone_Circuits>

<comment><USER6> And the base concept is probably based on Dwarf Fortress, where logic _does_ come from an unexpected and novel aspect: hydraulics.

</comment></comment></comment></comment><comment><USER7> Give people limited tools capable of building a turing-complete computer and eventually someone will... but it's always impressive to see it done.

Personally I have a railway to my underground tree farm and lava storage facility, but that's not exactly newsworthy :-)

</comment><comment><USER8> The ultimate end of all this is when he uses Minecraft to compile and run Minecraft.

</comment><comment><USER9> I'd be really excited to see Notch implement some sort of API or bridge to RL within the environment.

Then it's only a matter of time before people could offer Heroku-style SaaS deployments on MineCraft clusters for our future generations to enjoy.

</comment><comment><USER10> The book (from which he got the ALU specs) <http://www1.idc.ac.il/tecs/> and a Google Video based on it <http://video.google.com/videoplay?docid=7654043762021156507>

</comment><comment><USER11> See also: working computer made inside of Dwarf Fortress ([http://www.boingboing.net/2010/04/15/colossal-turing- mach.ht...](http://www.boingboing.net/2010/04/15/colossal-turing-mach.html))

</comment><comment><USER12> The creator talked about it on Reddit: [http://www.reddit.com/r/gaming/comments/dk1au/hey_rgaming_i_...](http://www.reddit.com/r/gaming/comments/dk1au/hey_rgaming_i_just_finished_an_alu_the_math_part/c10rnwj)

<comment><USER13> It's also worth noting that he basically got a job offer right there on Reddit.

<comment><USER14> How much would they pay a guy that makes 200k a day?

<comment> brown9-2 The offer was to the maker of this ALU, who is not Notch (the maker of Minecraft). These are two different people.

</comment></comment></comment></comment><comment><USER15> Simply amazing.

I really can't imagine how he wants to implement the compiler he speaks about :)

<comment><USER16> As it get it, it already exists - since the machine is a standard, it should be able to use already existing compilers.

</comment></comment><comment><USER17> I've never really had any interest in how an ALU works. I use one every day, and I take it for granted. But after seeing what an ALU looks like, in video game terms, the scale and complexity becomes apparent. Now I'm absolutely fascinated.

</comment><comment><USER18> Unless I misheard him, he made a working implementation of the Hack ALU, which is magnificent all by itself.

Hack is a virtual fully working computer that students build completely from the NAND up during a semester: [http://www.catonmat.net/blog/video-lecture- from-nand-to-tetr...](http://www.catonmat.net/blog/video-lecture-from-nand-to- tetris-in-12-steps/)

</comment><comment><USER19> _Are you listening, pig?_

For a second there I thought he was talking to me.

</comment><comment><USER20> The pigs made me laugh. I don't know much about minecraft, but I wonder if the pigs can foul up the circuitry. You don't get the right answer so you have to go clear the pigs out of the pathways. It's like the original legend about how software "bugs" are named so because a bug crawled into the circuitry and fried it out. In Minecraft computers, the bugs are called "pigs".

<comment><USER21> The pigs can't hurt it. But there are animals called 'creepers' that could mess it up quite badly. Only if you're close, though.

<comment><USER22> SSSSSSSssssssssssss

</comment></comment></comment><comment><USER23> Given a complex enough sandbox game and enough time, could one build a computer and version of the game which runs inside the game?

<comment><USER24> Yes, but it will be slow as hell.

</comment></comment><comment><USER25> It's amazing. LOL when I saw the pigs jumping!

<comment><USER26> while your comment is probably not appropriately worded for this community here, I have to agree: The pigs jumping around in the machine gave the video a new surreal dimension.

Consider all the work that went into this creation.

Finally - after everything is finished and everything works, the author takes his time to show us around. He's showing all the little details. All the ingenuity in the various components.

And then, in the middle of the tour, you see pigs jumping around.

I don't know. Maybe I have a simple mind, but I was highly amused - <USER2> knowing what a piece of art the creator of the video was showing us.

</comment></comment><comment><USER27> Also, in the video-game LittleBigPlanet someone made a functional calculator: <http://www.youtube.com/watch?v=ZiRgYBHoAoU></comment>


Joanna Russ' How to Suppress Women's Writing Is Being Reprinted - <AUTHOR> https://utpress.utexas.edu/books/russ-how-to-suppress-womens-writing <comment><AUTHOR> You should read it if for no other reason than that it is funny.</comment>


Developer of Tweetie launches LetterPress iPhone Game - <AUTHOR> http://www.atebits.com/letterpress/#

<comment><USER2> Further confirmation that search is completely broken in the App Store. Read about this app on my work computer, fire up the App Store on my phone, search 'Letterpress', Apple's Cards app, and a Letterpress Handbook about actual letterpresses.

<comment><USER3> It usually takes a while before new apps can be found with the full text search.

</comment></comment><comment><USER4> I find his pricing strategy for the game rather interesting. The game is free to play and contains no ads, but for $0.99, you can upgrade to a "full" version and unlock a few extra features. I think this model will definitely attract a lot of people, but I'm curious as to how many people will actually decide to pay for that upgrade (I did after about 15 minutes with the game ‒ I love it.)

<comment><USER5> From 2% to 4% used to be typical for shareware games (albeit on PC), in my experience.

<comment><USER6> I wouldn't be surprised if it were 5x or even 10x that. You've got one-click purchase, and it's only $0.99 - much easier upgrade route than on PC.

</comment></comment></comment><comment><USER7> It's interesting that an experienced app developer chose to create a game instead of a new app. Especially since the game category in iOS seems to be so saturated with free or freemium options.

Of course, I'm also surprised that experienced developers are still cranking out timer apps and weather apps. I guess it's just hard to come up with a good original app idea... much easier to fix what's wrong in the apps you use already.

</comment><comment><USER8> Cool game. The design Looks like a windows 8 app

</comment><comment><USER9> There's a nice interview with Loren about development and design of the game here: <http://www.youtube.com/watch?v=2VbqBBiEWu8>

</comment><comment><USER10> I love the UI (as I expected) but I feel like the market for these types of multi-player apps is waning.

Best of luck, though!

<comment><USER11> We really need a robust version of Matchmaking that games like Halo have so that you're never without an opponent and a good connection.

<comment><USER12> Until Letterpress I had no use for Game Center at all, but I just used its auto-matching to start two engaging rounds with strangers. Fun game.</comment></comment></comment>


HTTPS Results in 7% Google AdX Revenue Drop - <AUTHOR> http://blog.rome2rio.com/2016/04/18/https-results-in-7-google-adx-revenue-drop/ <comment><USER2> I saw an instant 30% drop in revenue when switching my site to HTTPS in April. The implementation was done right, A+ rating from ssllabs, Google reindexed my main pages as HTTPS within a matter of hours, search traffic and overall traffic remained unchanged.

I poked around on my AdSense account to see where I was losing the revenue, since AdSense was still displaying the same number of impressions. It turned out I was seeing a 75% drop in CPC impressions, and AdSense was running low paying CPM impressions instead.

[http://i.imgur.com/acy2k0u.png](http://i.imgur.com/acy2k0u.png)

That's a graph of daily CPC impressions on my account. It's obvious when I switched to HTTPS. That was over a month and a half ago. It hasn't bounced back.

I'm faced with a difficult decision now; whether to go back to HTTP and inform the community we're going to a less secure system for increased ad revenues, or I need to accept a 30% drop in my yearly income, and hope the situation improves as more networks switch to HTTPS.

<comment><USER3> >whether to go back to HTTP

If you do this, be careful of the HSTS header. You'll want to remove this from responses for a while before you flip everyone back to HTTP because their browsers will refuse to send HTTP requests during the specified period or until the next time they clear their browsing data.

If you don't have a lot of returning visitors, it matters less, but still something to be aware of.

<comment><USER4> Removing the HSTS header won't help, as your browser will store this data, and IIRC it's not stored in cache, so clearing won't help.

You can however send the HSTS header with max-age=0 which will start clearing it for existing users

<comment><USER3> Good tip about max-age=0! I didn't realize you could do that.

</comment></comment></comment></comment><comment><USER5> How does this work (economically)? Based on this subthread: [https://news.ycombinator.com/item?id=11804689](https://news.ycombinator.com/item?id=11804689) it seems that the mechanism is:

HTTPS site can't load resources over HTTP (due to security) -> only those advertisers who support HTTPS can place bids for HTTPS-only sites (and not all do) -> the smaller number of bids results in lower ad revenue

However, as also noted there, the advertisers who do support HTTPS should preferentially bid for impressions on HTTPS-only sites, since the price per view (or per click etc), is lower. Importantly, they should continue doing this (raising the price of adverts on HTTPS-only pages, due to the increased demand), until the price per click etc. is the same for HTTPS-only and "normal" sites. Why does this not happen?

i) The simple model above is overly simplistic, and I don't actually understand the situation,

ii) The marginal costs of serving HTTPS ads and HTTP ads are not the same (though why would that be — the server overhead for HTTPS is marginal) or the the revenue from HTTPS vs. HTTP ads is lower (but again, why?),

iii) The time-frame is too short(?),

iv) Advertisers are turning down free money (highly unlikely).

<comment><USER6> Isn't it simply a question of competition? If some advertisers can't support HTTPS, that means there are fewer bidders for HTTPS content. That should result in a lower price.

They are also bidding on what's presumably a shrinking pie of HTTP ad space. If the volume available to buy has dropped considerably, that may also be driving up the price of HTTP ad views.

<comment><USER7> Yes. But this very effect should drive https adoption for the advertisers. The grandparent comment was exactly wondering why this doesn't seem to be happening (enough).

</comment></comment></comment><comment> blubb-fish I think this is b/c almost all ad tags are loaded via HTTP.

The browser won't hand over the refer(r)er information from HTTPS to HTTP!

This lack of information causes a lower valuation of the inventory b/c the third-party ad networks (which are determined via AdX by an auction) don't know what they are bidding for.

I'm not exactly sure about the precise causation but I think this is the reason for that drop.

</comment><comment><USER8> If this is true, then advertisers have a temporary incentive to support HTTPS: it is currently relatively cheaper for them until more advertisers also support HTTPS.

</comment><comment><USER9> can someone explain why ad networks seem to be lagging behind modern web best practices? For example, terrible flash ads, ads that break the site they're displayed on, lack of HTTPS, malware, etc? And also why a company with a better engineering team hasn't been able to successfully take advantage of these weaknesses and corner the market?

<comment><USER10> The venn diagram of people caring about web best practices and people running adblockers is a circle, so there's no reason for ad networks to change anything.

<comment><USER11> To be fair, this ignores the thousands of software engineers in ad tech.

<comment><USER12> Having worked in ad tech, I didn't know anybody who wasn't running an adblocker in their primary browser.

<comment><USER13> I heard that Google had to rename some CSS names on their ad management portal (that understandably had names that began with "ad") to unbreak it when Google customers using adblockers were trying to manage their own ad accounts.

</comment></comment></comment></comment></comment><comment><USER14> HTTPS also increases the page load time, and there's plenty research that shows the relationship between latency and CTRs.

I'd like to add a shameless plug by noting that I'm working on a project that renders performance and security much of a false dichotomy within native iOS applications. We hope to incentivize the capitalists in the room to secure their applications by coincidence of making them faster. Caffeine: [http://www.caffei.net/](http://www.caffei.net/)

</comment><comment><USER15> Maybe a significant percentage of ad clicks are fraudulent and "more sophisticated" technologies like https break malicious traffic? I couldn't come up with alternative plausible explanation.

I believe this situation also shows weird dynamics with fighting adfraud. In general too much fraud can crash ad click economy, but on micro scale a lot of actors indirectly benefit from it so they don't have incentive to fight with it strongly. Especially if most automated way of fighting bots will also affect legitimate traffic (e.g. captcha for real users).

I would say real world analogy is oil prices and OPEC. Or more classical analogy is prisoner's dilemma. Individual actor incentives are different than best solution for the whole ecosystem.

<comment><USER16> I don't think fraud is a driving cause here.

The real cause is downgrade protection. When your browser loads an HTTPS page, it will refuse to load (or at least warn when loading) any other resources over HTTP: js, css, iframes, etc.

This is to ensure that the icon you see in your URL bar is actually accurate: if a page loads over HTTPS, but consists entirely of a single HTTP iframe, that nice green lock is totally meaningless.

So when the OP switched their site to HTTPS-only, they lost the ability to ever display any from HTTP-only bidders. So the set of people bidding on their ad slots went down, the price went down, and their revenue went down.

<comment><USER17> That is an excellent and counterintuitive but logically sound market-response explanation for the behavior.

I'm trying to think of how Google can attempt remedying this, with the more obvious options being to either increase the non-HTTPS search penalty, or perhaps to buy up some of the ad stock itself.

</comment></comment></comment><comment><USER18> I read a while back that something like for every $3 in ad revenue, $1 of that was fraud.

Maybe https is just breaking some fraud bots?

<comment><USER19> I'm convinced that on what _I_ spend, the fraction is way more than 1/3; I think it might be north of 2/3\. The clicks on and off the page are just way too quick and consistently so for it to be a human. That said, I'm not sure that's what's happening here.

<comment><USER20> Au least on mobile, often I will accidentally hit an ad, and immediately back out as soon as possible

<comment><USER19> Yes- mobile is a disaster. I can't imagine anybody profitably advertising on mobile. I've asked this group before and basically nobody has ever deliberately clicked a mobile ad (one guy said he did once, iirc). I have a hunch that the people making money off of these ads know this, too. It might be good for branding, but in that case you should only consider CPM-- CPC is your enemy here.

As I gear up for an advertising campaign, I'm looking pretty closely at channels.

</comment></comment></comment></comment><comment><USER21> Has the same thing happened for ad revenue using Admob in iOS apps since Apple made https communication the default? I have noticed a drop in revenue, but I assumed it was just seasonal, maybe there is a technical reason. Cheers.

</comment><comment><USER22> This has influenced me on my prediction for ([http://www.metaculus.com/questions/164/](http://www.metaculus.com/questions/164/)).

</comment><comment><USER23> Could some ad partners be holding out because http is getting them lower CPC for their customers or am I misunderstanding something?

</comment><comment><USER24> Wait, I don't get it.

The blog post explains what's happening but not the why. How could HTTPS possibly lower ad revenue? What is the mechanism?

<comment><USER25> > _We learnt that not all advertisers that place bids through the Google ads > system support HTTPS, resulting in fewer ad impression bids and lower > overall ad revenue._

<comment><USER24> I'm sorry, but can you walk me there? I know _very_ little about how Google ads work.

If I buy an ad from Google, I supply my own URL, which can be http or https, and if my site is https it can't display the http ad?

<comment><USER26> No, its that sites served over HTTPS won't show HTTP-only ads. Google determines who gets the view based on a auction system. Limiting supply of ads limits competitive bidding.

<comment><USER13> So Google does not serve the ads from its own servers? Google just runs the programmatic auction and then lets the winning advertiser point to whatever slow, non-HTTPS ad server they like? I guess once Google gets paid by the auction winner, they don't care what happens next with the served ad. And serving it from their own servers would just be an extra cost.

<comment><USER27> No, they don't actually serve all ads. I imagine they know not to pass off a request from a HTTPS page to a HTTP-only server (mixed-content blocking would give a near-100% failure rate) but here is an example of Google's Doubleclick network being used to spread malware: [https://blog.malwarebytes.org/threat- analysis/2014/09/google...](https://blog.malwarebytes.org/threat- analysis/2014/09/googles-doubleclick-ad-network-abused-once-again-in- malvertising-attacks/)

</comment></comment></comment></comment></comment></comment><comment><USER28> Why not offer both, HTTP (default) and HTTPS. Many websites work like that just fine.

</comment><comment><USER29> Is there a way to get around this through a proxy endpoint on your server?</comment>