How Bad Are Zillow "Zestimates"? - <AUTHOR> http://blog.duvora.com/exactly-how-bad-are-zillow-zestimates-case-study/ <comment><USER3> I am quite disappointed in the research that went into this article.

(1) Author discovers that Zillow has a specific claim about accuracy: "around 90% of the listing Zestimate is "Within 20% of Sale Price" of the home". Author then gripes that this is "a really large margin for error". Really? Because I was prepared to praise Zillow for such honest and clear reporting. It only costs a few hundred dollars _per home_ (Zillow covers nearly every home in the US) to get a professional to produce an assessment, and these assessments are usually only a little more accurate than what Zillow is claiming.

(2) Author pulls up tables from Zillow showing a median error. Which is an excellent way to analyze this and is (I think) extraordinarily small. Author then proceeds to do other stuff.

(3) Author decides to verify the values by checking a total of NINE houses. Is the accuracy of that test expected to impress us? Furthermore, one of these 9 is rejected because "(obviously a data glitch or input error. Dismiss this result)" \-- but that's the whole POINT of using median error; it isn't affected by outliers.

(4) Author finds that all of the values (all 3 of them) in Staten Island were pretty far off. Author asks professionals who say that the local market o Staten Island is behaving quite unusually at the moment. Reporting on this is the only thing I think the author did right.

<comment><USER4> A long time ago I got a real estate appraiser's license. Here is the procedure for appraising real estate, as taught by my state:

1) Pick three homes that you feel are comparable.

2) Pick out the material differences between those homes and your client's

3) Make up a value for each of those differences, and adjust accordingly

(Unofficially speaking) If you don't like the number you come up with, just make different decisions in steps 1-3 until you get the number you want.

<comment><USER5> In the late 90's I worked for an online mortgage broker, and 9 times out of 10 when we ordered an appraisal, the appraiser would ask us what the selling price was and return an identical figure for their estimate.

<comment><USER6> Amazing that people still pay for that.

<comment><USER7> "People" don't. It's a required part of the due diligence to get a mortgage. No appraisal, no loan.

For the most part, the banks and brokers re-sell those loans immediately, so they don't stay on their books. All they need to protect them from lawsuits later is proof that they followed the process to the letter -- which involves getting a written appraisal.

No mortgage broker or Realtor ever got rich by picking a fight with the appraiser to lower the appraised value ... there is literally nobody present at the table who has an incentive to push the value down.

<comment><USER6> I understand that it's required, and that organizations often follow processes long after they're revealed to be foolish.

But it's been a decade now since the housing bubble revealed that appraisals are meaningless.

Why has no one eliminated them?

<comment><USER3> They are required by law or by long-standing policy in a government-run body (which is practically the same thing). Laws DO change to keep up with the times, but they do so rather slowly. In 50 to 100 years, if there is continuous clear evidence that home assessments are highly unreliable and not particularly profitable to anyone, then you should expect them to be eliminated.

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> I don't have an opinion on whether zestimates are accurate. But this blog post has 9 samples from only 3 cities. And the author seems to be making claims based on that. This is a lame "thought leadership" post as a subtle ad for his company.

At least give me 10-20 homes in each of 10-20 cities if you are trying to convince me of something systematic.

<comment><USER9> The only really interesting thing I saw in this post was the discussion with realtors. It's significant that Zillow uses long-term averages, and therefore can't price in abrupt market shifts even once they're common knowledge. If nothing else, it implies that "Zestimates" are a bit dubious in tight urban markets where high variance is more possible. Of course, it also looks like Zillow is aware of that challenge.

The ostensibly data-driven part was by far the least convincing section - <USER2> soon as I saw the sample size I decided it was nothing more than a starting spot-check.

<comment><USER10> Perhaps Zillow is more interested in showing the longer-term market value of a home so buyers know if they're being asked for a high demand-driven price. A tight market like that can spring up quickly and fade away just as quickly. At some point most people are going to look to resell or to at least not be forever upside down because they bought during a tight market.

</comment></comment></comment><comment><USER11> SF Bay area, I've had 3 formal house assessments, & each time they "start with" Zillow et al to look at "reasonable values in the neighborhood". Which seriously affects your ability to refi, etc.

So Yeah Zillow has an effect. It's worth it to create a login & enter in your upgrades, so your ZESTIMATE goes up.

<comment><USER12> _It 's worth it to create a login & enter in your upgrades, so your ZESTIMATE goes up._

Very nice, thank you, my Zillow estimate is now $5965 higher. We have no intention to sell but if we do, at least I can start the negotiations at a higher price point by pointing to Zillow.

<comment><USER13> Incredible -- I just did the same, and my house's "zestimate" went up about $20,000.

Of course, if I compare my house to actual sales in the neighborhood over the past few months, that's probably 10% high, so correcting Zillow's data about my house made the zestimate less accurate, not more.

<comment><USER14> Try changing the number of bedrooms. That used to have an effect, even for the same square footage. In one case, 2 bedrooms came out higher than 3.

</comment></comment></comment></comment><comment><USER15> Realtors who feel threatened by Zillow love to complain about Zestimates. Zillow has made it clear time and time again that they are not 100% accurate and even allow homeowners to edit them.

Zestimates are a tool that homeowners like to use. Zillow knows that. Good Realtors won't feel threatened by it but instead leverage it to provide prospects with a much more complete and accurate estimate.

Realtors need to stop blaming Zillow & Zestimates for their own weaknesses. It is getting old.

<comment><USER16> Zillow deserves a fair share of the blame because they prominently feature a single number with dollar-level precision and bury the caveats multiple clicks away on a separate page.

Some basic UI changes could avoid all of that: e.g. instead of showing $123,456, show a range like $115,000-135,000, and keep the last few digits at zeros based on their estimated error rates. Similarly, their graphs could use shading to make it clear that they're showing the mid-point of a range rather than a discrete value.

<comment><USER17> > Some basic UI changes could avoid all of that: e.g. instead of showing > $123,456, show a range like $115,000-135,000

So true. When you know that you are not displaying an exact dollar amount you should not use exact dollar values. Exact dollar values afford exactness even though the branding is an estimate.

<comment><USER18> This might very well backfire on them. The buyers will want to buy at the lower price and the seller would want the higher price. Both parties end up being unhappy even if the end price was within the given band.

</comment></comment></comment></comment><comment><USER19> I bought a home last year, and each realtor I talked to brought up how terrible Zillow was before I ever mentioned it. If you look at the WaPo article all the comments are by other realtors, so I'd take those comments with a grain of salt. Oddly, my mortgage guy never looked at Zillow (that I saw) for figuring out values, so _shrug_.

I'm sure the data is inaccurate, but it's somewhere to start the conversation and let people feel like they have some power in the process of buying and selling homes.

And as people like to say about Uber, realtors are welcome to make a competing site that's as free and easy to use.

<comment><USER20> My bro-in-law became a real estate agent a couple years ago. Recently we were discussing Zillow, and he went into a tirade about how terrible it is. I just mentioned that it's a good tool to gauge the price trends of my neighborhood, but he began countering with canned, carefully scripted responses that sounded like indoctrination from his broker/employer. I just sighed, before the topic turned to how important it is to pay the full 6% when listing your house.

The real estate industry is one of the most protectionist industries out there, and any threat of disruption, no matter how minor, is met by an army of vitriol.

<comment><USER21> What were the arguments for the 6%? There is a broker in the Bay Area who advertisers that they take half that but the complaints against them see to point to the fact that you'll only get offers from other affiliated agents, and many agents won't share the listing with clients if they can avoid it because they want their full commission.

I'm the Bay Area I feel like a seller's agent is grossly overpaid for what is often a weekend of open house showings and then fending off multiple above list price zero contingency offers.

<comment><USER20> I've sold houses using discounted brokers and he hates that. He said that buyer agents will dissuade buyers from looking at houses offering less than "standard 3%" to the listing agent (another protectionist tactic).

I countered that I don't care. In this day and age buyers are savvy enough to jump on Zillow and Redfin and find my house, and they will demand they see it. If I hired a buyer agent and they tried to talk me out of showing a house over a commission I'd fire them right there.

</comment></comment></comment></comment><comment><USER22> Zillow's Zestimates are particularly obnoxious because they provide little detail regarding why the values change and they retroactively revise them. I bought a house this summer that had an inexplicable drop in estimated value of about 15% a month or two before the house was listed. I paid closer to the previous estimated value than the post-drop value. Now when I look at the house on Zillow, the drop is nowhere to be seen and the past year is a smooth, barely sloping line. Never was there any indication of _why_ they had a 15% drop, which was particularly frustrating when trying to determine how much to offer for the house.

<comment><USER23> This has happened to me as well.

</comment></comment><comment><USER24> Interesting. While I agree the results are likely to be inaccurate, this case study targets only the three largest cities in the US. Urban centers probably aren't the best population from which to sample.

In a case like this, we'd probably want to use locations based on number of home sales; and even then, we'd need to account for the amount of time between homes changing hands i/e houses that are flipped. There's likely something to be said about the difference between the nation's average sale price and each area's average sale price; assuming that may affect Zillow's estimations.

Anyways, I like that you're doing studies like this, regardless.

<comment><USER25> I'd guess part of why those cities were chosen were to have enough data points to draw conclusions, since all the examples are from a narrow time frame.

</comment></comment><comment><USER26> There are ~100M homes, ~3k counties, ~27k zipcodes, and ~300k neighborhoods and subdivisions that all have different market dynamics. A sample size of 9 make this analysis completely meaningless.

Equally important, is that the author of this article doesn't appear to know much about how the real estate data industry works (especially in regards to market trends and how they are used to generate the Automated Valuation Models (AVMs for short). A bit surprising given the nature of the business he is representing.

The Zestimate is one of a handful of Automated Valuation Models (AVMs in industry speak) - it's the only one with a consumer-facing brand, and as a result sees the most scrutiny despite the fact that the other AVMs on the market are the ones actually used by the banks during the appraisal process.

If you think the Zestimate is bad, you should see some of the other commercial AVMs.

The reality is that creating an automated valuation for the ~100M properties in the US is an incredibly difficult task given the availability of data (or lack thereof!). Zillow does a pretty darn good job and has a bit of a data advantage given its incredibly high coverage of for sale listings (though they lack the experience - some firms have been producting AVMs for decades!).

Another thing to note, is that AVMs (like any statistical model), usually aim for a sweet spot that covers as many homes as possible. As a result, ultra high-end properties are often incorrectly valued. E.g. that $25M mansion down the street will be very hard for the model to price. Additionally, certain luxury features, like say, a tennis court, are almost impossible to incorporate into these models on a national scale.

</comment><comment><USER27> Better question is "how bad are the compared to those of professional appraisers who look at half a dozen comprables?" An even more important question is "how biased (based on who is paying) are those professional appraisers?"

<comment><USER13> Isn't it strange how the appraisal always seems to come in just a little above where it needs to be in order for the financing to be approved?

<comment><USER28> An appraisal is telling you what the market will bear for a particular property. What better indication of the market value than the most recent price agreed to?

<comment><USER27> You're missing the point. May appraisers are under the thumb of the mortgage brokers. See [http://www.nakedcapitalism.com/2014/12/bill-black- mortgage-a...](http://www.nakedcapitalism.com/2014/12/bill-black-mortgage- appraisal-fraud-baack-bank-execs-profit.html)

[I]n 2007the New York State attorney general sued First American: relying on internal company documents, the complaint alleged the corporation improperly let Washington Mutual's loan production staff ''hand-pick appraisers who bring in appraisal values high enough to permit WaMu's loans to close, and improperly permit WaMu to pressure …. appraisers to change appraisal values that are too low to permit loans to close" [FCIC 2011: 92].

</comment></comment></comment></comment><comment><USER29> Considering a Zestimate doesn't take into account anything about the aesthetics or actual character of the house, being off by 20% doesn't seem like it is that much. People respond to more than just the pure square footage and neighborhood. The Zestimate should be taken as nothing more than a directional starting point for someone who doesn't have the time to actually look at neighborhood comps.

<comment><USER30> Isnt that what it is marketed as? I found it useful as a starting point when shopping.

<comment><USER29> Exactly. I think the writer is taking issue with trying to use it as a more exact science, and calls it out for being within 20% of the sale value 90% of the time, which I think is pretty reasonable considering the high amount of subjectivity in home purchasing.

</comment></comment></comment><comment><USER31> I looked my house up on Zillow and it is showing an estimate of about $80k. My homeowner's insurance has it covered up to $75k, the county tax assessor lists the FMV as about $50k, and I owe $37k on it. My own estimate is about the same as the tax assessor's, and I wouldn't ask more than $50k when it comes time to sell it, given the work it needs to get it up to the Zillow estimate in my eyes.

Looking at other houses in the area, my house appears to have a much higher perceived vs actual value, based on Zillow estimates alone. For example, my neighbor's house on Zillow is supposedly worth $90k, but I'd put it closer to $100k based on her lot size, house size, overall condition, and age of the house (I'm friendly with my neighbor so I know a little about her property's history and makeup). In other words, her Zestimate is a bit lower than what a reasonable person would put it at, whereas mine seems greatly inflated. Perhaps this is due to Zillow "grading on a curve"; it would seem odd if one house was rated at half the value of its neighbors, so mine gets brought up to the lower end of the average range even though it doesn't deserve it. That's purely speculation on my part though.

<comment><USER32> Offtopic, but I had to chuckle a little: Bay Area house prices have gotten my mind so clouded that I instantly thought you were missing an extra zero on each of those dollar values. I had to re-read it twice until it dawned on me that there are other places out there...

<comment><USER33> Same thing happened with me. Coastal housing prices skew your perception of reality so much, it's hard to imagine <$100K even existing.

</comment></comment></comment><comment><USER34> While we're at it, lets also figure out why my Zestimate dropped $40k overnight two months ago. If I was planning to sell, I would have a huge problem on my hands. On top of it the Zestimate chart shows no indication that they dropped it $40k. Aiming to fix the number is a fool's game : my "private home estimate" on my owner dashboard is 23% higher.

Its a worthless number, and they should stop trying to manipulate the market. Having a secret measurement formula that works for 70% of the houses but not the other 30% needs to either have a disclaimer that its all fantasy, or just outright be removed. I'm guesstimating those 70/30 numbers, but their site gives some locations 2 stars on their own scale: [http://www.zillow.com/zestimate/#acc](http://www.zillow.com/zestimate/#acc). If its not 4 stars, then don't publish it.

<comment><USER35> From time to time, Zillow rejiggers (that's a technical term) their algorithm, resulting in sudden readjustments of estimates. Given that the Zestimate itself is presented as a fairly wide range, the error margin is hardly hidden.

Also, it is important to remember that no appraiser from Zillow has actually looked at your house; the data is drawn entirely from sales data, public records, and volunteered information. So there's no way it can really account for every factor, nor does it even try. And there's a lag; the market can change faster than Zillow can. It is, at best, a rough ballpark estimate of what's going on in your neighborhood.

<comment><USER34> point taken, but two observations:

1) its ridiculous to also "fix" the zestimate history. Put the old one in the history, and explain near the chart what rejiggered.

2) The market in my immediate area, as evidence by the comps from zillow and the sales prices in the newspaper, has only increased. The conservative comps are near the old zestimate - this is a suburban area. The wild ones are higher than my private estimate.

<comment><USER36> I think what you're actually seeing is the output of a Kalman Filter temporal model. The zestimate is the mean of the filter output, but the error bounds of the filter are more important than the actual mean/reported value.

What happens is that the model uses trends to extrapolate from each "real" data point (in this case, a house sale in your neighborhood). The problem is, and what Kalman Filters help manage, is the uncertainty propagation between each house sale. When it has been a long time from when a sale has occurred, it is unclear what the real/actual price is of a home. This means that on the estimated price the error bounds are large, and zestimate still just reports the mean value of this huge uncertainty.

What then happens is that a house is sold in your area, a new data point is recorded, and the filter re-adjusts itself and collapses its uncertainty/error bounds in the time of that measurement around the measurement. And you get correction. This is why the "old zestimate" is updated.

</comment></comment></comment></comment><comment><USER37> More comprehensive studies have ben performed: [https://www.washingtonpost.com/news/where-we- live/wp/2014/06...](https://www.washingtonpost.com/news/where-we- live/wp/2014/06/10/how-accurate-is-zillows-zestimate-not-very-says-one- washington-area-agent/)

I just bought a home myself and looked this up, but the takeaway is that the Zestimate is within +- 5% 50% of the time. So, not that accurate. Are people really using it in negotiations though? I'm sure people do, but you'd have to be pretty uninformed to do so, and I'm also sure you're real estate agent would tell you exactly that up front.

<comment><USER38> Yeah, but how accurate are appraisals? Several years ago I was reading about an analysis that found that Zestimates were about as accurate as appraisals are 80%+ of the time.

<comment><USER39> The job of the appraiser is to write down a number as large as or larger than the amount to be financed. They know this and they know the number in advance. What they do is not rigorous.

<comment><USER40> Definitely not true... there are many checks in place to ensure this doesn't happen

[http://www.freddiemac.com/loanadvisorsuite/loancollateraladv...](http://www.freddiemac.com/loanadvisorsuite/loancollateraladvisor/) [http://www.freddiemac.com/loanadvisorsuite/faq/loancollatera...](http://www.freddiemac.com/loanadvisorsuite/faq/loancollateraladvisor.html)

This doesn't even take into account the process for appraisal management companies (third parties).

<comment><USER39> You seem to think these industry controls are effective. Maybe they prevent an appraisal from being, say, 10x higher than the recent sale prices on either side, but they don't prevent the equally bad event of a series of 10 sales on the same block each being 10% higher than the previous. Every one of those houses in Las Vegas that turned out to be worthless in 2007 was appraised for at least the sale price the year before. The appraisers in those cases didn't do a damn thing, for example they didn't come out to the house and write down "this neighborhood is in a desert with no access to water where the unemployment rate is 20% and the employed people make minimum wage therefore this cannot be a neighborhood of $850000 houses." They just said "the last house sold for $800000 and this market seems pretty hot so $850000 okey dokey."

</comment></comment></comment></comment></comment><comment><USER41> I use Zillow to give me a ballpark number for prices in an area I don't know well. My experience is that they are usually within 20% of the market price, which coincidentally they say on their website. For a free automated service, Zillow is great. If you want a better number, do your own comp analysis, or hire an appraiser.

</comment><comment><USER42> I've been working in the mortgage industry for the past few months and have learned a bit about the business.

What Zillow does with its Zestimate product really isn't that novel. There is a whole class of products available called AVMs (Automated Valuation Models) which are used in the industry which pre-date Zillow's Zestimates. What Zillow did which was innovative was brought this type of product to the general public, for free.

In some cases an AVM is all that is needed to secure certain types of loans such as HELOCs.

</comment><comment><USER43> Redfin does a much better job with estimates and track their error rate: [https://www.redfin.com/redfin-estimate](https://www.redfin.com/redfin- estimate)

</comment><comment><USER44> I find Zestimate _over time_ to be useful. I've been watching my Arizona property climbing up since 2010 when it was estimated at rock bottom, or about 50% of its purchase value (sigh). Now it's estimated to be about 90% of its pre-crash value. But as with all private transactions, bottom line it's worth what someone's willing to pay for it.

</comment><comment><USER45> In my experience, Zillow does a terrible job compared to local knowledge. My zip code is weird—our ~$275k house backs up to houses worth $50-70k (the disparity is due to build quality, lot size, and the streets they're on). Not surprisingly, Zillow handles this very poorly, and has consistently valued our house at much lower than it is actually worth (based on city assessments and sales of nearby comps).

Interestingly, Zillow recently bumped up our Zestimate from $190k to $230k. What's puzzling is that they also bumped up the _historical_ Zestimates—there isn't a big jump shown on the graph on their site (though there is a big jump shown in my Personal Capital account, which uses Zestimates as part of their net worth scores). Either they modified their algorithm in a way that affected values retroactively, or they're trying to disguise the fact that their previous Zestimates were way off.

<comment><USER46> > What's puzzling is that they also bumped up the historical Zestimates

There are two things a "historical estimate" could mean:

1) What did Zillow say the value was on day X?

2) What does Zillow now think the value was on day X?

You and others seem to be thinking of (1), but I think Zillow actually is doing (2). Since the goal is to figure out what the house is worth and how its value has changed, I think (2) is actually the right metric here.

<comment><USER47> (2), of course, has the advantage of obscuring past mistakes and producing a nice smooth chart.

<comment><USER48> Unless you think they should never update their algorithm, approach (2) is just making the chart consistent with itself.

The useful part of these charts isn't really the current Zestimate, but rather the trend line of the home values, especially at the neighborhood level.

If you don't recalculate the historical values, then you break the ability to see the trend line.

If you really want to see the results of previous models, then the way to do that is to plot a separate line for each model, but you can see why they wouldn't want to do that from a product perspective.

<comment><USER47> Every county in the US that publishes historical valuations seem to get by without overwriting history.

<comment><USER48> Yes, but those are actual valuations based on sales prices, not predicted valuations based on an algorithm. Different thing.

</comment></comment></comment></comment></comment></comment><comment><USER49> 5M home sales yearly, so this shouldn't be done by anecdote.

<comment> ctrl-j That was my first thought as well. Analyzing 3 houses in 3 markets... It's an interesting look, but I wouldn't consider any takeaway message to be necessarily accurate.

Algorithms have outliers, and quantitative analysis requires adequate sample size.

</comment></comment><comment><USER50> Whether or not the data collected in this article is questionable, there's a pretty easy way to evaluate the price of your home: re-finance / hire an appraiser. If you believe your home to be worth more, wouldn't it be worth the $200 or so to increase the value of your home by _thousands_ by having an accurate market research from a qualified appraiser? I got mine as part of my refinance package, so I have a very accurate estimate of what my home is worth, comped out against other homes in my area that sold recently at a similar location/sq ft/etc. Not sure why you feel the need to use questionable data points like Zillow in the first place.

</comment><comment><USER51> In New York Zillow seems to be very dependent on property tax assessment data, at least in my region. They also don't have a good way to distinguish between neighborhoods well.

That said, they do a pretty decent job overall. Property valuation is art and science, and is probably a lot easier in a place like LA with lots of tract homes built in similar timeframes to similar spec.

I live in a city founded in the 1600s. Housing within a 5 block radius is built between 1890 and 2016. It's not a market with a lot of flipping, so it's hard to value as a house that looks like mine (reasonably modern internals) may have 1960s kitchens and bathrooms.

</comment><comment><USER52> A very smart statistician who worked for Amazon once said, "estimates are useless without error bars". The problem with Zillow is they really should just show a range, not a fixed number that appears to be accurate to the ones place.

Edit since I wasn't clear: I know they show a range in tiny text, but they main number they show should be the range if they want to avoid all of the "is Zillow accurate" articles.

<comment><USER53> Except Zillow actually _does_ show a range. If you look at the Zestimate details on a house, you see something like this:

"Zestimate $472,603", and immediately to the right: "Zestimate range $425k - $506k". The little question mark reveals:

"The Value Range is the high and low estimate market value for which Zillow values a home. The more information, the smaller the range, and the more accurate the Zestimate. See data coverage and accuracy table."

</comment></comment><comment><USER54> Zillow isn't accurate for Austin.

Using two identical homes in my neighborhood about 3 doors apart from each other, mine / neighbors.

Zillow - $387,546 / $384,326

Redfin - $417,808 / $420,737

TCAD - $368,876 / 372,802

(Recent Appraisal) $427,000 / (Recent Sale) $429,000

What's worse, is that when I contested my taxes Zillow used the TCAD numbers to lower the value of my home on their site. Uh... Zillow, you know it's in our best interest to contest our taxes, right? Has nothing really to do with the value of our homes... it's a mix of us wanting to pay less and the City / County wanting us to pay more.

I'm sure there are more scientific studies, but for my house I'd say Zillow is ~10% under-market. When I went in to contest my taxes this last year, I was told by a friendly TCAD employee that Redfin numbers were better and that I should use Redfin when trying to cite examples of comparable properties with lower taxable values.

Keep in mind most of the value depends on your realtor's ability to show the house, how well you stage your house, and the number of of interested buyers.

</comment><comment><USER14> Thought exercise: if you were Zillow, what additional data would you need to make the estimates more accurate? And where would you get it?

</comment><comment><USER55> This article deserves the heavy criticism it has received here. I'll go further and say it is part of a trend you might call skepticism gone wrong. This is a intellectual movement that "throws the baby out with the bathwater" because they are so skeptical that they don't believe anything!

Many authors in this vein have read about, for example, cognitive bias, the many invalid results in scientific journals and the hoax/fake news/infowars trend. They see _every_ human action as a result of bias, every result as invalid, and every news article as part of a misinformation campaign.

Having the most accurate possible understanding of the world we live in is only possibly by trusting information sources which deserve it, understanding the limits of what "quick and dirty" statistical analysis is capable of, and recognizing when people are influenced by bias, and also when people are acting with full awareness of bias.

</comment><comment><USER56> As far as I can tell, Zestimate works by taking prices of recently sold nearby houses and then estimate gain over previously sold prices or the property tax estimates. If Zillow wasn't in the picture, you would likely do the same thing. This obviously would generate inaccurate estimates if house wasn't in market since many years or if nearby houses are too different or too few. Another source of inaccuracy is that they don't seem to take in account school zoning very well. So house in good school area can inflate price of nearby house which doesn't have same school.

In my experience, Zillow has played big part in house pricing. Seller in hot market will almost always set the max of Zestimate and price in their mind. That prices decreases only slowly over time if buyers don't show up. In hot market usually some sucker will show up anyway and Zestimate becomes self- fulfilling prophecy.

</comment><comment><USER57> Zestimates are only reliable as an 'in the region of $X' indicator. Seen multiple houses where this issue exists. I'm sure those are not the only ones.

I know Zillow doesn't make a claim about being super accurate, but not everyone is an intellectual buyer doing math in their head. For a lot of people, they treat zillow as a roughly true indicator and are surprised when that doesn't match reality. Or they completely dismiss it in the very beginning and rely on their agents or intuition.

You'd be surprised how seriously people who aren't quant inclined can take a prediction or estimate. Well, given that all the quant inclined among us believed Hillary would win, it's obvious that humans tend to put trust in other people's estimates if they essentially trust those people would do a good job unless proven otherwise. It reduces the cognitive burden of calculating it ourselves.

</comment><comment><USER52> > If you excluded trust sales and probate sales or family-to-family type Quit > Claim transfers (where the property is transferred at a considerably lower > value from its market value) shouldn't the Zestimate reset to the exact > amount of the property's last sold price?

No, it should not. My personal example: When I bought my house, I bought it for 20% less than what Zillow said. But Zillow would have been right on the money _if my house were in average condition_. But it wasn't, it was unliveable. The first thing we did, before we moved in, was spend 9 weeks renovating it. Even then we didn't even bring it up to the local average. It took us five years to finally finish all the work.

Zillow estimates assume the house is in average condition, because they have no way of knowing otherwise.

<comment><USER14> I wonder if you could estimate the condition of the house based on a street- view picture of the outside. There is probably correlation between outside upkeep and interior condition.

<comment><USER52> Maybe. Ours looked ok from the outside, was definitely worse on the inside.

Incidentally we bought it from the original owners who had owned for 55 years. I wonder if there is a correlation between condition and how long the current owner had it.

</comment></comment></comment><comment><USER58> I bought my home for 139,000 4 years ago. The bank appraised it at 155,000 while i was buying it (also 4 years ago). The town has it appraised at 178,900 since 2011 Zillow's "Zestimate" has it at 134,500

If we go by the "market" rate of what it would/could actually sell for then Zillow is the most accurate out of all those estimates.

Now I've done quite a bit of interior work to the house and Zillow's is most likely off by quite a bit now that my house has a new kitchen and bathrooms but I'm not sure how it could take that into account since there's no way for that data to reach Zillow.

</comment><comment><USER59> I don't know how accurate these things are, though my gut tells me not to trust them. That said, as a recent home-buyer, I can state without uncertainty that real estate agents hate Zillow, and Zestimates in particular.

<comment><USER60> I concur. Though I tend to find this attitude pretty ridiculous since the status-quo home valuation process is just as bullshit at the zillow strategy is.

Status Quo in my experience goes:

1\. List your house (usually at slightly above whatever the realtor thinks it might sell at)

2\. Someone's willing to buy at listing price +/\- 5%

3\. Bank sends an appraiser whose job it is to figure out what a house is worth (guess what: see #2 for relevant demand evidence), and magically the number comes remarkably close to the price sorted out in #2 massively most of the time

I've never heard a real estate agent complain that zillow's numbers are too low, mind you... only ever that they're too high, which makes sense to be frustrated by if your biggest challenge is controlling your time-costs.

</comment></comment><comment><USER61> Really dumb but fixable flaw in the algo: I bought my home from a mother and daughter who each owned half. Town hall filed the two transactions on the same afternoon.

Zillow counts it as two data points at half price!

It's the only sale on the property so they have two half price "historical" inputs to the Zestimate, and that makes the real listing price seem nuts to an Internet home buyer.

They refuse to correct the data, insisting it sold twice at full price on the same day instead of two halves, and saying it's on me to get the town hall to reflect it as a single transaction, which it was not.

</comment><comment><USER62> Given that the company's own CEO didn't use his own company's "Zestimate" when it was time to zell his house, I think the accuracy of the number is fairly perceived to be zub-optimal.

"You gotta eat your own dog food," goes the old zaying.

[http://www.inman.com/2016/05/18/zillow-ceo-spencer- rascoff-s...](http://www.inman.com/2016/05/18/zillow-ceo-spencer-rascoff-sold- home-for-much-less-than-zestimate/)

</comment><comment><USER63> It's not suprising that estimates based on "user-submitted data points" are not accurate. Everyone who ever tried to sell a house knows how difficult it is to estimate real market price of property. Usually people will submit higher valuation of their house so that they can negotiate it down in case their valuation is not realistic. It is better to give higher value so that you have room for negotiation. Transaction prices will always differ from listing prices.

</comment><comment><USER64> Just over two months ago, I sold my house within $1,600 of the "Zestimate". I then turned around and purchased another house over 10% below its "Zestimate"

</comment><comment><USER65> If you have a SFH try this out. Way better than zillow. [https://www.housecanary.com/](https://www.housecanary.com/)

<comment><USER66> Which house canary product does free home valuations?

<comment><USER67> We have free trials of our valuation product, Value Report: [https://www.housecanary.com/product-value- report](https://www.housecanary.com/product-value-report)

<comment><USER68> When do you plan to add support for multi-units? I have a duplex...

<comment><USER67> We're working on getting all of our single family residence products where we want them before we move into multi. I don't have any timeline for when that would happen.

</comment></comment></comment></comment></comment><comment><USER69> I bought a house a year and a half ago; Zenstimates are consistently ridiculously low for my area. I was looking for single family homes in Somerville, MA. There is so little supply that every home goes immediately after listing and most likely with a bidding war. If you look at the Zenstimate for my property, it looks like I over-payed by almost 30%, but I guarantee I could flip for a profit tomorrow.

</comment><comment><USER70> The Zestimate for my house is incredibly close, within $2k of a recent appraisal. There are a lot of comparables in the area, so they have a decent amount of data to work with.

I have a friend who just bought a 3,000sf house on 1/2 of an acre in Lexington, VA for $31k. Zillow has it at $400,000. He's either incredibly lucky and should buy me a beer, or Zillow is the worst. Leaning towards the latter.

<comment><USER71> Like the previous reply, I'm going to assume you meant $310k, not $31k, as $31k for that property doesn't make sense based on the factors you listed unless the place is a dump.

As to the disperity, this could be easily explained by seller factors. Maybe the house is actually worth $400k, but they _needed_ to sell it, even if less than it was worth (had a contract on another home, home was a foreclosure, etc etc).

<comment><USER70> I did not drop a zero. He paid cash.

<comment><USER72> is it a nuclear waste dumping ground? What's the mitigating factor?

<comment><USER70> In the country and needed about $10k of renovations. Sold as-is. It _may_ end up being worth $200k, if he's lucky. Zillow is just incredibly far off.

</comment></comment></comment></comment></comment><comment><USER73> At some point my home was converted to a duplex, and converted back into a single family home before I bought it. I've tried multiple times to get Zillow to fix the listing but they've been "unable" to do even such a simple thing...

I suspect their data quality is pretty terrible if they're unable to do any curation, so it's no wonder the zestimates are all over the place.

</comment><comment><USER74> Anecdotally, I've noticed that Zestimates are biased by location.

According to Zillow, 90% of the houses in one region were overpriced and selling like hotcakes. And 90% of houses in another region were underpriced but staying on the market.

I think most of the value is not an absolute measure, but as a relative one. I can compare Zestimates to each other in the same city with fair accuracy.

</comment><comment><USER75> I don't know about "zestimates", but they seems to not be able to do something as simple as email address verification properly. Admittedly, based only on the five email's I have received from realtors in North Carolina just this morning (I live in Nebraska and have no plans to move any time soon).

</comment><comment><USER76> I'm still scratching my head and wondering what Zestimates mean for rentals.

For example, I'm giving serious thoughts to moving to SoCal, and I see a place that looks pretty nice... it's listed as being $2000/month, but then when I go to the listing it says Zestimate $2650/month. What's up with that?

<comment><USER68> Is $2000 within the rent range? Is it a duplex? Seems like Zillow data is far more accurate for single family homes.

</comment></comment><comment><USER77> Wait - is the author just measuring Zillow's accuracy the same way Zillow does? As in, "Here's a poll that asked N people something. Let's check the poll's accuracy by calling 3 of those people and asking them the same question again."

That seems... not useful in any way.

</comment><comment><USER78> Zillow is beyond useless. My property consists of 4 legal "lots", two of which the house physically occupies. Each lot is assessed separately by the gov't for tax purposes. The "Zestimate" considers only the lot associated with the street address. So about 40% of the property valuation is ignored.

Second, my house is lakefront. Lake frontage is probably the single biggest factor in valuing the property. Yet Zillow ignores lake frontage as a feature. In fact a recent check of so-called "comps" included no lakefront homes at all. A complaint to Zillow was met with a yawn and a "well its just a number people can use as a starting point, and real estate professionals in the area will know the particulars." So basically they told me the "Zestimate" has no value and they know it.

<comment><USER79> I have a similar situation with my condo in brooklyn, which has a deck and a great view, which doesn't count towards square footage. So the zestimate is way lower than what we paid.

We recently moved, and rented our condo, and we were able to rent it for 2k more than the rental zestimate. So, I agree that it's to be taken with a grain of salt, and not that important.

</comment></comment><comment><USER80> This article failed to compare Zestimates to any of the numerous competing estimation services and also neglected to mention that the primary competition for these computed estimates is intuition of Realtors and brokers.

</comment><comment><USER81> Ohio, residential home, Zillow = $310,000, sold for $210,000

</comment><comment><USER82> In the hot west coast markets, Zestimates are usually fairly accurate but often end up low compared to closing due to bidding wars and mania.</comment>


How Zapier made us support a hundred new services in half an hour - <AUTHOR> http://ducksboard.com/blog/zapier-support-hundred-new-services

<comment><USER2> The title sounds like a negative, like Zapier adjusted their service and all of a sudden Ducksboard had a lot more work to do.

Maybe "How Zapier allowed us to support a hundred new services in half an hour"?

<comment><USER3> Ah, the ambiguities of the English tongue and the unaware non-native speakers that get ensnared by them!

Anyway, I hope the intended meaning gets through and it was also a case of "shit, people, we _need_ to be on this thing!". So they made us support new services as gas makes a car run, and they also made us do it as a madman makes Sandra Bullock go over 50 mph.

Disclaimer(s): I wrote the original post and I had some beers.

</comment></comment><comment><USER4> This is an awesome anecdote. If anyone is in the process of building an open REST API for their company right now, shoot me an email. We'd love to help kick-start your platform.

<comment><USER5> Yes - we're building a big integration with lots of ecommerce platforms and would love to find a way we can work together. I'll drop you a note.</comment></comment>


The CONS microprocessor (1974) [pdf] - <AUTHOR> https://dspace.mit.edu/bitstream/handle/1721.1/41115/AI_WP_080.pdf <comment><USER2> It's interesting to see that, although what we call now a "microprocessor" had already been invented, in that paper they use the same word to describe a CPU implemented using microcode (and not a very small processor that fits in a single chip).

Here is a picture of the author of that paper working on the CADR machine (the successor to the CONS machine): [http://www.computerhistory.org/chess/stl-431614f64ea3e/](http://www.computerhistory.org/chess/stl-431614f64ea3e/)

<comment><USER3> Here's a link to the CADR paper: [http://www.unlambda.com/lispm/memo528.html](http://www.unlambda.com/lispm/memo528.html) I like these; they describe a microcoded machine in more detail than I can usually find.

</comment></comment><comment><USER4> A little background here - Lisp is an ideal language for genetic algorithms because its syntax maps directly to S-expressions which can be represented as a tree (warning, PDF):

[http://www.cis.umassd.edu/~ivalova/Spring08/cis412/Ectures/G...](http://www.cis.umassd.edu/~ivalova/Spring08/cis412/Ectures/GP.pdf)

This tree can be directly represented as a binary string which represents the DNA of the genetic algorithm.

So evolution is easy to simulate by randomly mutating nodes in the tree. Other operations like crossover (exchanging branches) and sexual reproduction are also straightforward.

Personally, I think that genetic algorithms are more approachable than neural nets and would like to see them emphasized more in machine learning (especially with modern video cards). When you think about it, both approaches are searching for local maximums/minimums in an effectively infinite search space. They are basically solving a giant matrix to find the function that maps inputs to outputs. Sure genetic algorithms generally take more computing power, but that becomes less of a problem with each passing year. Given enough time, evolution always wins!

</comment><comment><USER3> Not much about Lisp here. It describes the design of a microcode-based machine that could be used for all sorts of things. You could use it to implement a PDP-11, an x86, or whatever (though it's biased towards 32-bit machines).

Google "microcode" and you'll get a start. Hennessy and Patterson will carry you further.

</comment><comment><USER5> I will admit that this stuff is way over my head, but it's things like this that make me wonder why the Lisp machines/processors never really caught on...can anyone here shed some light into this for me?

<comment><USER6> Wikipedia has a little snippet on this: [https://en.wikipedia.org/wiki/Lisp_machine#End_of_the_Lisp_m...](https://en.wikipedia.org/wiki/Lisp_machine#End_of_the_Lisp_machines)

Basically, lisp machines were expensive workstation computers, and PCs killed all workstation manufacturers. Sun SPARCStations and Silicon Graphics machines bit the dust as well. Nobody could justify the cost anymore.

<comment><USER7> >lisp machines were expensive workstation computers

In most cases, yes, but the language also found its way into some very unexpected places. For example, a subset of Common Lisp, "L", was at the core of Prof. Rod Brooks' Subsumption Architecture which enabled the amazing mobile robots built in his lab. He ran Lisp on 16mhz 68332 embedded processors. [https://www.researchgate.net/publication/2949173_L_-- _A_Comm...](https://www.researchgate.net/publication/2949173_L_-- _A_Common_Lisp_for_Embedded_Systems)</comment></comment></comment>


A 'Darker Narrative' of Print's Future From Clay Shirky - grey-area http://mobile.nytimes.com/blogs/publiceditor/2015/04/10/a-darker-narrative-of-prints-future-from-clay-shirky/ <comment><AUTHOR> Does anyone know the history of technology stacks used by NYT to provide comments? Like HN, the comments are often more insightful than the article:

      Transactional revenue

_"... Yes, "think of subscribership as membership." But that doesn't mean "in short, get some percentage of the loyal readers of The Times to pay more — some of them a lot more." It means, leveraging the strength of the very real community of loyal readers that the Times has, by building more and more revenue streams from delivering more and more value to members of that community."_

      Public debate

_"... A good place to begin would be with the comments areas, with the goal to transform them from the current depositories of opinion into well moderated forums for dialogue which instead of producing points of view stimulate learning and advancement. Public intellectuals rather than know-it-alls would be the result. That would be different!"_

      Weekly cultural ritual

_"... The Sunday edition of The NYT is different. I peruse this all week while giving my electronic devices a break. I'm older and still enjoy the serendipity of discovery. Yet I rarely look at the A-section. It's old news and I've already digested all - more than likely I have digested it prior to your pages being sent to the printing plant."_

 _" Truth be told, I'd be sad about the disappearance of all but the Sunday edition from print, but more to the point: It's infuriating, and confounding, that the Times insists on undermining its Sunday edition, a cultural institution so important and dear to its readers' hearts."_

 _"... every Sunday morning I would buy the $$$Sunday London Times PAPER, spending all the long day reading it because every article in that preposterously-thick paper was delicious-even the sports section._"

<comment><USER2> Their blog comments are far better than their article comments. I think it's more about audience than the technology.

<comment><AUTHOR> It would be interesting if the non-paying audience contributes higher quality comments :) Or maybe lower-traffic blogs attract less/different posturing than comments on high traffic articles.

</comment></comment></comment><comment><USER3> Shirky's post in 2009 was one of those great clarion calls. Failure is always an option (and not the kind you learn from and pivot off of).

Noting that newspapers aren't necessary (but journalism is) is great, but the problem is that a lot of (most?) journalism is still done in newspapers... and the way the media is currently breaking up, where companies like the Tribune Company and Gannett are splitting their television holdings from their newspapers means that journalism and newspapers may not be inherently tied together... but when it comes down to it, they are still for all practical intents and purposes tightly coupled.

Web based news orgs like Vox, Fusion, Gawker and Buzzfeed are making some moves towards funding journalism (Buzzfeed in particular has great investigative reporters), but they are way too few in number to replace newspapers as a home for investigative journalism even as the industry stands today in its depleted state.

<comment><USER4> Here's a question: How much money goes in to hardcore investigative journalism?

<comment><USER3> That's a hard thing to disentangle, because investigative reporting isn't just done by people with "investigative reporter" in their titles.

If you want a random waypoint Propublica's staff costs for 2011 were around 5mil$ for a small but very investigative focused news org: [https://projects.propublica.org/nonprofits/organizations/142...](https://projects.propublica.org/nonprofits/organizations/142007220) (granted they're also not a typical news org).

</comment></comment></comment><comment><USER5> "2) Do more to cut costs, companywide. ("The most valuable long-term dollar to an organization with declining revenues is a dollar you don't spend.")"

This one worries me. I don't have better proposals to save the industry, but we keep redefining coverage to be less direct and less expensive and we all end up paying for this by being further removed from news as it happens.

As a digital-only subscriber, I find NYT's current subscription model bizarre. For the base ($20/mo) tier, I get access to web and iPad but not iPhone (which costs an additional $10). This structure means I use and value the service less, making me more likely to cancel. Feels like a business model that privileges their needs over their customer's experience. And as a result, I opine that it is not likely to meet their expectations. I'd rather see an attempt to broaden the subscription base by aggressively lowering the cost ($7.99) and aggressively expanding the entitlement (sub includes crossword, for example).

</comment><comment><USER6> We (my partner and I) subscribe to the Sunday only print edition, which gives us digital access. We shouldn't really be considered "print subscribers" in a traditional sense. 90% of the time we scan the magazine and maybe read one article in the paper. Then straight to the recycle. The majority of our consumption, and engagement with ads, happens throughout the week as we share a single digital account. The physical paper is more of a souvenir, or possibly a house decoration, than a means of engaging with the stories in the paper.

</comment><comment><USER7> Also by Clay Shirky, and very much worth reading:

Healthcare.gov and the Gulf Between Planning and Reality

[https://news.ycombinator.com/item?id=6770039](https://news.ycombinator.com/item?id=6770039)

</comment><comment><USER8> I'm waiting for the killer e-ink devices. I love my reader and am anticipating it's evolution will eventually make it a staple for most adult millennials. Once that thing is crystal clear, fast, cheap (and perhaps can be folded in half), I find it hard to believe it won't rejuvenate the newspaper market.

<comment><USER3> better e-ink devices won't fix the problem that the internet has completely changed the advertising market and how brands reach consumers.

It's hard to see what newspapers could do to get the same returns per unit sold or page viewed as they have historically. And that means a couple possible things: readers bear more of the cost, advertisements get more intrusive (or crazier), or newspapers find some other way to make money.

<comment><USER9> Personally, I would love to pay more for magazines and newspapers without any advertising whatsoever. Advertising always seems to have some influence on what is reported. A newspaper or magazine will be very reluctant to say anything very negative about the companies/industries that advertise with them, because they might lose those ads then.

Then there's the problem of ads influencing me directly, causing me to spend more money on other things, than I otherwise would.

I guess advertising in general, can cause reporters to have an attitude of "if you have a problem, buy a product to solve said problem" rather than "if you have a problem, here's some knowledge to solve said problem".

So overall, I would probably save a significant amount of money by paying more for advertising-free media. But most people don't seem to understand this.

</comment></comment></comment><comment><USER10> I wonder if there is any data on the fixed costs of newspaper production - digital or not? That is how much do they spend getting words to the bifurcation point - where the words could go to a printing press or to a web server?

That seems to me to be the important part. The words and knowledge of the NYT is the important bit - whether I read them on paper or screen seems less important.

And if they cannot afford to produce good words for a subscription only clientele, perhaps they need to find ways to monetise "influence". Pay us and we will keep influencing politicans on your behalf - essentially expand lobbying out past the proprietors.

<comment><USER11> I've seen some back of the envelope numbers [1] but not good data. It does seem pretty clear though that, in contrast to eBooks, printing is a pretty substantial part of the overall costs. I've heard it said that, as a rough rule of thumb, subscriptions plus single-copy purchases historically covered the physical printing and circulation of the paper while advertising covered everything else (plus profit).

The problem is that you can't just move ad revenue from print to digital.

[1] A few years ago and Business Insider but FWIW [http://www.businessinsider.com/2009/1/printing-the-nyt- costs...](http://www.businessinsider.com/2009/1/printing-the-nyt-costs-twice- as-much-as-sending-every-subscriber-a-free-kindle)

</comment></comment><comment><USER12> One idea is to monetize stories submitted by PR firms [1] and clearly label the stories as such. At the same time, keep or increase the quality of such articles to make it worth the reader's while.

To capture more of the value, the newspaper industry should also create a PR platform (comparable to AdWords) that allows firms to directly and efficiently pitch their products and their social/business impact to the editors. (Cutting out the middleman) It of course should remain at the editor's discretion whether and in what form to print the stories.

Increase in transparency (that the story originated from a PR pitch) and monetization opportunity (to pay for investigative reporting) should benefit both the reader and the newspapers.

[1] [http://www.paulgraham.com/submarine.html](http://www.paulgraham.com/submarine.html)

<comment><USER11> It's called native advertising and it's somewhat controversial. The problem is that, even if it's labeled, it erodes the wall between advertising and editorial. Many magazines have had "special advertising sections" forever, but the more you blend in native advertising, the more readers are likely to confuse your editorial with an advertiser's pitch.

Of course, especially in some areas, PR narratives do drive the choice of story (as suggested in your link) a lot of the time. But it's still a slippery slope to rent out column inches for content that purports to be a form of editorial.

<comment><USER12> Well, the idea is to make sure the article quality be on par with other editorials. This should include critical commentary and comparison of the product or company in question to its competitors. This would be something quite different from those special advertising sections, which almost always feature only its positive aspects.

Since newspapers already publish these articles anyway (without labeling the PR source), the slippery slope is already in action. Although I do agree that money could make things worse for some papers, market mechanisms and transparency could help balance things out.

</comment></comment></comment><comment><USER13> This interview with Tyler Brule of the very successful Monocle Magazine, is relevant and worth a read as he's been a booster of print. He draws the same conclusion as point #3, that the iPad wasn't going to be a saviour, and I think he would lean to #4 (subscription as membership) as being the best path forward. At least that seems to be the direction Monocle has moved in.

[http://www.niemanlab.org/2015/03/how-tyler-brule-has- extende...](http://www.niemanlab.org/2015/03/how-tyler-brule-has-extended- monocle-beyond-simply-a-magazine-for-the-jet-set/)

<comment><AUTHOR> These points stuck out from the interview:

      Print-on-demand news

_"..Monocle newsstand near London's Paddington Station. The kiosk will offer a curated selection of magazines and newspapers as well as global newspapers printed on-demand. Customers will be able to register online, request a printing of newspapers ranging from Norway's Aftenbladet to The Australian, and then come to the newsstand to pick it up ... "It's important for us to, I think, remind the market that — is it really a crisis of print, or is it a crisis of print distribution?"_

      Trend chasing

_"..We don't have an issue of some digital guru or a friend or an investor or a board member who says, Why aren't you slapping that little bird on the bottom of every webpage and every printed page? ... we don't have that influence surrounding our brand, that we have to follow the dictates of a big parent corporation, and that's been the great thing from the very start."_

      The companies they keep

_"..the power of adjacency, of being in with a core print product group beside an ad for Cadillac, beside an ad for Cathay Pacific ... Here [his BlackBerry again] I can go and buy space in like-minded editorial, but I'm never going to be surrounded. I'm never going to have that pure adjacency right next to another brand in the same way that I would in print ... Brands want to have good adjacencies the same way that companies will wait forever to get space between 57th and Madison and 5th — because they want to be in the right company. I think that's still very hard here."_

</comment></comment><comment><USER14> In my opinion media companies should primarily cash on loyalty, rather than readership. Special editions with unique content, merchandise, and above all, events. Events add a ton of value to attendees and is very lucrative opportunity. Doesn't matter if we talk about industry conferences or TedTalk- like type of event. Attendees get a ton of value out of it and are willing to pay a lot for it. Same goes for sponsors.

It works out great for industry-based media. I don't see why it wouldn't work for mass media organizations.</comment>


Why Google Needs To Keep Murdoch - <AUTHOR> http://pagesaresocial.com/2009/12/01/why-google-needs-to-keep-murdoch/

<comment><USER2> This article is a classic false dichotomy: Pitting Fox against blogs. In reality, there are literally thousands of news outlets world-wide covering every major story from every angle. If Fox were to disappear from Google, who would even notice?

Checking out Google News right now, there are 9,881 articles on President Obama's recent speech. Fox is just one voice amongst thousands. And ultimately, that's what Murdoch is truly upset about; news is an easy obtained commodity and is therefore rapidly losing its worth.

<comment><USER3> Yeah, exactly. I wonder if that's why Fox features so much insane commentary? News is fairly easy to get, even primary sources are easy to get, but a community of like-minded people is still worth something.

I bet people watch Fox because they want to hear the commentary of people who share their political beliefs, not because they want to know what's going on in the world.

Kind of like reading Hacker News for the comments.

<comment><USER4> Absolutely spot on.

</comment></comment></comment><comment><USER3> This would only make sense if Fox were the only trusted news outlet (to borrow the article's term) that Google indexes. Since it's one of a whole bunch, Fox is really only hurting themselves here.

People will go to Google to search for a story, they'll get a bunch of hits to news sources that are not Fox (and are not blogs, to keep this author's mother happy) and will read those instead.

The choice isn't "Fox or a bunch of blogs", it's "Fox or CNN/BBC/every-other- news-outlet-on-the-planet".

</comment><comment><USER5> Let Murdoch throw his temper tantrum. Google should black-hole every one of his sites.</comment>


Deepfake Videos Could Destroy Trust in Society – Here's How to Restore It - <AUTHOR> https://liwaiwai.com/2019/06/24/deepfake-videos-could-destroy-trust-in-society-heres-how-to-restore-it/ <comment><USER2> Who needs Deepfakes? There are images from 2015 circulating Twitter today that claim the current administration is putting kids in cages.

It just states it as fact with no proof and thousands of people believe it.</comment>


Graydon Hoare: What next for compiled languages? - <AUTHOR> http://lambda-the-ultimate.org/node/5466 <comment><USER2> Related discussion here: [https://news.ycombinator.com/item?id=15051645](https://news.ycombinator.com/item?id=15051645)

</comment><comment><USER3> At first I thought this was a [dupe], but at closer inspection it's a discussion page on the lamdba language site, showing how that community addresses the topics.

I think this is an important article/discussion to promote on HN, but the title might draw people away from it due to its similarity to the referenced post.

<comment><USER4> Am I reading that page right in that it (currently) contains three comments, where only one of them is somewhat substantive? I feel like I'm missing something.

<comment><USER5> This should practically fall under blogspam. Up next: posting old links to hacker news instead of reposting. Or linking to reddit threads instead of their destination</comment></comment></comment>


Ask HN: Can we finish with the Apple 3.3.1 Policy submissions? - <AUTHOR>

Apple made a business decision, and the development community needs to make theirs.<p>As we have seen, there have been good arguments both for and against.  Developers will vote with their feet, and I'm sure Apple thought through that risk.  No matter how many posts, upvotes, downvotes, etc. I don't think Steve Jobs is going to change his mind.  I feel we are missing other good submissions with all the focus on things beyond our control and which are becoming philosophical discussions. <comment><USER2> This is how a news site like this works. If somethings get popular a lot of people will get interested in the topic and post stories about it. Then more people get interested in the topic and so on. It's also a reflection what other sites write about. I don't think somehow limiting how long a topic can be discussed will be good, as a lot of insightful stories will be posted some time after an event occurred. It will fade naturally with time as it becomes less topical.

There's always going to be people that don't find a certain topic interesting, but if something is upvoted it means that other people are. So just read the other stories that you feel interested in and when something you are interested in comes up, others can return the favor.

Also, I do think there are some "less than optimal conditions" at HN, especially for new stories. But I don't think something becoming "overly" popular is because of those conditions.

<comment><USER3> So many stories are being posted that people aren't reading all of them, leading to the same arguments being posted repeatedly.

</comment></comment><comment><USER4> Apple setting a bad precedent with this was my hope. I'm now 50% closer to my eventual wish - Apple exerting too much control and screwing up. The control part is done. The screw up... fingers crossed. That will discourage other companies (at least in this space) from doing it. Developers FTW... I'm hoping.

</comment><comment><USER5> It's one of the hottest stories we've ever seen on HN and I tend to see 'quantity of stories on X' as one indicator of where X will be going in the near future. Simply scanning through the HN headlines without reading every individual post has value for me.

</comment><comment><USER6> Is the fixation on this topic a bug in the HN software or in the people reading HN?

<comment><USER7> I think it's just that plenty of people so far on the fence or on the side of apple in all the app-store and DRM conversations have suddenly 'seen the light'.

<comment><USER8> Personally, I've considered, being long time Linux user, purchasing something with OSX on board.

Thankfully, Mr. Jobs corrected my path. And, probably, some people other people too.</comment></comment></comment>


Impossible - <AUTHOR> http://ninjasandrobots.com/impossible

<comment><USER2> I feel like the kitchen can be an entrepreneur's playground. Cooking (and eating) was always a very social affair with my friends and family. I started learning to cook as a child when I helped my mom and dad make Sunday dinner for a family of 6. I often cooked for myself when I went to college. I even cooked big Sunday dinners and charged guys in my dorm $5 bucks. I guess you could say this was my first venture.

<comment><AUTHOR> Exactly. I think it's an incredible place to remove yourself from the grind of starting/running a business, but still work on something creative and inspiring.

<comment><USER3> Definitely agree with this. I, personally, do quite a bit of mixology and cooking as my non-computer based creative outlet. I get a lot of enjoyment out thinking through various combinations of ingredients and how they would interact with each other to create something new and interesting.

</comment></comment></comment><comment><USER4> I've always seen constraints as a way of propelling creativity. Constraints basically stop us from wondering about "what if I had.." or "if only I had.." and make us think in "with my current resources how can I achieve something?". This post proves it in the world of cooking but this translates perfect in any software related solution or any business.</comment>


Creating different storage formats in Quickstep - <AUTHOR> http://quickstep.incubator.apache.org/guides/2017/03/30/storage-formats-quickstep.html <comment><AUTHOR> Brief summary: This post gives a brief primer on various storage formats in Quickstep and how to use them in practice.

About the project: Quickstep is a high performance relational data processing engine. We are an Apache incubating project, which initially started off in University of Wisconsin - Madison. URL: [http://quickstep.incubator.apache.org/](http://quickstep.incubator.apache.org/)</comment>


"Tough. Django produces XHTML." - <AUTHOR> http://groups.google.com/group/django-developers/msg/a65fbbc8effcd914 <comment><USER2> _The fundamental problem here is the need to output in multiple markup languages. [...] However, Django simply does not work at the level of abstraction that would allow multiple output languages._

The Django people have always been quite up-front about their goal of providing a full-stack framework with all basic components included and tested rather than an all-singing, all-dancing, pluggable and composable architecture. I can't say what they've come up with is to my personal tastes but it's a perfectly reasonable design choice. "What real world problems would such a change solve" is a similarly reasonable and pragmatic question. "Tough. Django produces XHTML" makes them sound like bigger jerks than they probably are!

<comment><USER3> I think that one of Django's nicest benefits is that even though it isn't all singing/all dancing, you can make it do just about any song or any dance with some extra code. I'm not saying that don't have valid reasons to do otherwise, but I think it would be more inline with the "django philosophy" if you could make the HTML generation extensible.

<comment><AUTHOR> It is and there are fixes out there for the HTML/XHTML thing but this is mostly an argument about defaults and built-ins.

</comment></comment></comment><comment><AUTHOR> Some context:

It's mainly about Django's Admin and form generation - in most other places Django is markup agnostic.

</comment><comment><AUTHOR> Some choice quotes:

"Switching back to HTML4 is driven by the same kind of purity-beats- practicality, fashion-conscious silliness that made us all switch to XHTML in the first place."

"But to worry about being able to instantly switch to the doctype-du-jour -- or rather _last years'_ doctype-du-jour -- _as well_ as having HTML validity - that's not being a perfectionist, it's called OCD, and I'm drawing the line there."

<comment><USER4> XHTML is last year's doctype-du-jour, too.

<comment><AUTHOR> He does go on to mention that 'HTML5 will save us' - at least from quibbling over
 vs
...

<comment><USER5> Unfortunately he thinks that he can't switch to HTML5 today and get all the benefits he requires. Do it, there's no reason not to.

<http://html5doctor.com/absent-elements-and-validation/>

[http://wiki.whatwg.org/wiki/FAQ#Should_I_close_empty_element...](http://wiki.whatwg.org/wiki/FAQ#Should_I_close_empty_elements_with_.2F.3E_or_.3E.3F)

<comment><USER6> actually, there is a reason. because in IE6 it breaks depending on context. if you have an incorrect content header and don't prefix <http://> on your url, then having an html 5 doctype makes your page try to "download" instead of display.

Whereas having a doctype that matches the tags you actually use, (xhtml or html4) will work fine without the <http://> prefix. As most people don't use that prefix when typing your url, this can have a drastic effect on your page views.

<comment><USER5> Have you got a link describing this issue?

I've not heard about it before and Google's not giving me anything.

<comment><USER6> afraid not, as I recently came across it and have fixed it by adjusting the content headers. it's a quick fix, but can be detrimental as a lot of web servers will present the incorrect content headers and developers will be completely unaware since browsers will obey doctypes over content headers (I think). In this particular case IE6 will obey the content header since it doesn't understand the html5 doctype and will try to "download" the content instead.

It should be trivial to setup a test case if you want. Setup a virtual host with an empty content-type html header, and a html5 doctype and open it in IE6 without putting <http://> before the url. You should see it try to download the page.

<comment><USER5> Not that I'm doubting you, but it doesn't really make sense to me.

Browsers sniff the doctype to change rendering mode, but that's after they've already decided that it's HTML that they're going to display.

IE6 & 7 do content type sniffing, where they ignore what the server tells them it is and try and figure it out themselves, by looking at various bits of info including the start of the file.

This could lead to HTML being downloaded if it happened to look like a RAR file to IE, but generally it has the opposite effect as in something being sent as plain text that happens to include tags being rendered as HTML.

If anything, shortening the doctype should make something look more like HTML since it works without any doctype at all and the shorter the doctype the more room for HTML tags.

On the other hand content-sniffing is always going to give you unexpected results. Is there anything about the particular file you had that makes it atypical or likely to look like a binary file if you only consider the start of it?

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> Serious question: what is wrong with XHTML? I've been using the XHTML 1.1 Strict DTD for the last 3 years straight, always creating fully validating code. I found XHTML infinitely easier to appease than HTML4. Perhaps not as important, but I've also found that I can create nearly identical XHTML in different Web frameworks (depending on the task), which is important to some of my clients who don't have a lot of resources to upgrade the plethora of legacy systems they operate. I don't mean that as a brag "I'm so great", to the contrary, I've just found XHTML to be much more consistent and far easier to construct via robot than HTML4, meaning I _don't_ have to be "so great" to use it.

<comment><USER8> See my comment here:

<http://news.ycombinator.com/item?id=1094925>

<comment><USER9> Yes but - it's still true (for me at least), that XHTML is far easier to construct automatically. Mostly due to the tag-closing behaviour.

I don't spend all day writing HTML, so it's not at the forefront of my mind which tags are self-closing and which ones aren't. Is it "
" or "
"? And really, why would I care? And ever more really, how much of that behaviour can I be bothered putting into automatically-produced output, rather than just simply putting the "/" everywhere?

If I cheat and serve up XHTML-like tags under a text/html mimetype, then nothing and nobody cares (as far as I can tell) except for the W3 validator.

<comment><USER5> You can't just put the / everywhere, and you still have to differentiate between non-empty and empty elements (e.g. div and br).

If you write XHTML, and don't follow these two non-XML rules (and indeed the other 14), then it'll probably break when you give it to an HTML parser or browser:

<http://www.w3.org/TR/xhtml1/guidelines.html#C_2>

C.2 Empty Elements: Include a space before the trailing / and > of empty elements, e.g.
,
 and . Also, use the minimized tag syntax for empty elements, e.g.
, as the alternative syntax

 allowed by XML gives uncertain results in many existing user agents.

<http://www.w3.org/TR/xhtml1/guidelines.html#C_3>

C.3. Element Minimization and Empty Element Content: Given an empty instance of an element whose content model is not EMPTY (for example, an empty title or paragraph) do not use the minimized form (e.g. use

 and not
).

</comment></comment></comment></comment><comment><USER10> If Django's XHTML-style form tags bother you (they bother me) you can always use my django-html template library, which fixes the problem in a kind-of-but- not-too-hacky way.

<http://github.com/simonw/django-html>

<comment><USER11> Why do they bother you? Its literally a matter of

        <input type="text" name="email" />

vs.

        <input type="text" name="email">

Why do you prefer the latter? (Genuinely curious, not being rude)

<comment><USER10> I'm a perfectionist.

</comment></comment></comment><comment><USER12> My plan has always been:

* keep making tags lowercase

* keep quoting attribute values

* keep closing tags after I open them (and properly nesting tags)

* keep using the closing slash for standalone tags

* keep calling it text/html (mime type)

* keep using UTF-8 encoded Unicode by default

* keep not using xml namespaces

* keep forgetting to use the DOCTYPE declaration

* and start carefully using some of the neat new tags in html5

<comment><USER10> Forgetting to use a DOCTYPE is a bad idea, since it throws most browsers in to "quirks" mode meaning they won't render things according to the published standards. That makes debugging problems a whole lot harder since you first have to work out if the bug relates to quirks mode v.s. standards mode, and quirks mode problems are far less widely documented.

<comment><USER12> Given my html as described above, what would you recommend to use for the DOCTYPE line?

<comment><USER13> <!DOCTYPE html>

easy to remember and triggers standards mode

<comment><USER12> Looks redundant, but easy. Thanks.

</comment></comment></comment></comment></comment><comment><USER14> I missed the memo. Can anyone point me to an article explaining exactly why XHTML has suddenly become the whipping boy du jour? I'm well aware that HTML5 is the next hot thing and all, but I don't understand why anyone would favor HTML4?

<comment><USER5> <http://hixie.ch/advocacy/xhtml>

_"A number of problems resulting from the use of the text/html MIME type in conjunction with XHTML content are discussed. It is suggested that XHTML delivered as text/html is broken and XHTML delivered as text/xml is risky, so authors intending their work for public consumption should stick to HTML 4.01, and authors who wish to use XHTML should deliver their markup as application/xhtml+xml."_

Note the date (Sep 2002) and the fact that the author is now in charge of the HTML5 spec.

Some people argue that forcing web developers to follow rigid XML rules, plus an appendix full of hints on how to mangle your XML so that actual browsers wouldn't choke on it (<http://www.w3.org/TR/xhtml1/guidelines.html>) somehow got people in the right mindset to do semantic, accessible HTML. The prevailing view is that this was mostly cargo cult behaviour and neither those following or recommending this course of action really understood the supposed benefits or drawbacks.

<comment><USER15> And some more:

[http://www.456bereastreet.com/archive/200501/the_perils_of_u...](http://www.456bereastreet.com/archive/200501/the_perils_of_using_xhtml_properly/)

<http://lachy.id.au/log/2005/12/xhtml-beginners>

</comment></comment></comment><comment><USER16> _You only have problems if you try to switch to application/xhtml+xml, which is easy to solve: don't do that_

 _if we were outputting HTML4, it might cause ∗genuine∗ problems - e.g. if you need XHTML in order to mix in MathML or SVG_

.. to do which you must serve as application/xhtml+xml.

That said, I agree with his points. And personally I wish the browsers would relax the restrictions on when they will agree to render SVG. I see no good reason why they refuse to do svg in text/html mode, considering the exceptions they make for seemingly everything else.

<comment><USER17> "I see no good reason why they refuse to do svg in text/html mode"

This probably breaks major assumptions inside the browser engines. Rendering most image types is a one-and-done type deal, just setting up a few calls to libjpeg or libpng, but SVG actually has to be XML-parsed and rendered element by element on page load. To do it in text/html, you'd need to hook together the html parser with the XML parser with the image generator. That's a lot of work to break the rules!

<comment><USER16> Well, good point, but I can't imagine it would be _that_ hard, and they'd have to do it anyway for HTML5, which does indeed support svg inside text/html.

<http://www.whatwg.org/specs/web-apps/current-work/#svg-0></comment></comment></comment>