Bye Bye Craigslist - <AUTHOR> http://blog.padmapper.com/2012/06/22/bye-bye-craigslist/ <comment><USER4> This is just offensive.

It would be one thing if Craigslist was doing their best to satisfy their customers and they wanted to shut down others who were scraping their site and leeching off their business. However, Craigslist puts absolutely no effort into making their product usable. The continued existence of Craigslist is just a testimony to the enormous strength of network effect lock in.

<comment><USER5> Disagree. Craig Newmark has said that he's scared to make drastic changes to the UX specifically because he doesn't know what magic recipe keeps people coming back. Do I like the CL layout? Nope. Does that mean other people should steal the content and profit from it? Probably not.

<comment><USER4> The magic recipe that keeps people coming back is that Craigslist has a large network.

I would be willing to bet a large amount of money that adding a map to search results and letting me put an upper limit on the number of bedrooms I'm looking for would not make people less likely to use Craigslist.

<comment><USER6> Their magic recipe is that an amazing amount of non-technical people have figured out how to work with the site.

The Dutch version of Craigslist (Marktplaats, owned by eBay) is in a similar predicament.

They have a very sucky UI and have experimented with new and improved versions, but every improvement was quickly reverted after large drops in engagement.

It's almost surreal to look at their site (<http://marktplaats.nl>) and realize that there's tons of highly talented people behind it.

<comment><USER7> Yup, there's a catch-22 here: when your site has shitty usability but is the only source of information, people hit more pages and spend more time there. When you fix your UI mistakes, they're in and out quicker because they actually get what they need efficiently. Because you're optimizing for engagement by measuring time on site or something like that, you end up making sure never to make your site easy to use.

This is not a recipe for success. It's a symptom of measuring the wrong metric, especially when you have a near monopoly on a market due to network effects. If this is really why Craigslist is not improving their UI, they need to hire some analysts that actually know WTF they're doing.

<comment><USER8> Haha, I see what you did there. You confused UI with Usability. Craigslist is incredibly usable. Have you taken a look at Ebay lately? Does it look familiar? That's cos it hasn't significantly changed in 10 years. Is that because they don't care? Nope, it's because gradients and drop shadows make your product look better, but don't make it more usable.

Craigslist is a utility, not a website. They care about keeping it useful not only for users today, but for users in 10 years. That's not something startups care about at all. Startups are looking to amass users and then sell. Craigslist cares not about such goals.

Go ahead, get mad at their inflexibility. But then take a step back and think about what craigslist represents. And then wonder if you yourself would have the cajones to stick to your guns in todays startup market. I for one commend their ability to keep their product strong in the face of all the bells and whistles of web 2.0 or whatever it's called these days.

<comment><USER9> Just because the site is simple in design, doesn't mean that it is built to be used efficiently. Craigslist, as is, is a usability nightmare. When I use the site, I am using it with various Greasemonkey scripts, Firefox extensions and 3rd party web apps to make their content easier to consume/see/understand/filter. These tools are the only reason that I can bare to use Craigslist.

If I am searching for a bike, a simple script to show image thumbnails by every listing will let me see which listings interest me. This saves me time, because now I only have to open a few interesting links. I don't have to scour through many of text links, just to see if they interest me.

The rentals section is constantly bombarded by spam and has very few filtering options. There is no way to exclude keywords. Boolean operators would greatly improve the usability of the site.

<comment><USER10> You can exclude keywords and there is a boolean OR operator: |

The problem is that you have to enter some search query before you can use the negation operator.

Also, protip to get past the spam in rentals section (Austin is holy crap bad about this): enter 'google' as your search term. This will only return ads that have an address specified and hence have the 'google maps' links. Plus, since you've entered a search term, you can filter out the locator who keeps spamming and screwing up your search by putting his phone number in the address section.

Here's a query I might have used during my last apartment search.

        google downtown | central -"512-555-5555" -"CALL NOW" -"austin apartments NOW"

etc.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> I am totally upset. I wanted to use padmapper to search for an apartment for my next move. I had been looking with it for months and wow it was super helpful.

Please, make a client-side solution so they can't shut you off! A browser based plugin, for example that allows us to plot the listings on our end.

I wrote this email to craigslist:

Hi Craig and Jim,

I am very upset that you cut off padmapper. Using padmapper cuts out a lot of the spam that ends up in the apartment listings due to the fact that it doesn't have a lat,long pair most of the time. It also makes it much easier to search for apartments based on your preference. With padmapper, I could see if the apartment I'm looking at is in a safe area, how far it is from a potential employer, and a myriad of other great things.

Finding an apartment with craigslist, in spite of the fact that you have the data, is much harder than searching on padmapper.

I really wish there was a way you could work with them!

You guys are a solid blue chip, no doubt. But if you take this oligarchical perspective to your business, someone is going to beat you eventually.

Thank you for your continued service

<comment><USER12> Great email... I will follow suit. They cut off the listings right in the middle of a search and it took me a little while to figure out what happened.

</comment></comment><comment><AUTHOR> Hi everyone, Eric from PadMapper here, thanks for all the comments!

To everyone saying that this is lifting their content, I disagree - it makes a summary of the content and then points back to the original. If I wanted to lift their stuff, I would have made separate pages that laid things out better than the original. Literally none of the text except the title and location string make it into the summary.

It's not a copyright issue, their main legal beef is that it's against their TOU for anything but general search engines to index their content - <USER2> vertical ones allowed. Who knows what their actual motivation is.

<comment><USER13> Do you think they're shutting you down in preparation of creating their own version of pad paper an releasing that without your competition?

<comment><AUTHOR> It's possible, they've been looking for a dev with maps experience for a while.

</comment></comment></comment><comment><USER14> For those wishing to take action:

email@example.com – Jim Buckmaster, CEO

email@example.com – Craig Newmark, Founder

@jimbuckmaster on twitter - <http://twitter.com/jimbuckmaster>

@craignewmark on twitter - <http://twitter.com/craignewmark>

Simple email to copy and paste:

Jim and Craig,

I'm a longtime user of Craigslist, but I'm really disappointed in your decision regarding PadMapper. It would be great if you could weigh in on the debate happening on Hacker News or elsewhere.

Thanks,

<comment><USER15> Good luck, but I doubt it will work.

My first experience with craigslist was trying to figure out why they had the capability to block my wife's ad for a free dog that contained a mispelled word while the site's creator was busy trying to tell the government that there was nothing he could do about the many child prostitution ads that appeared on craigslist.

<comment><USER16> Craigslist does not allow you to sell or give away animals -- it likely had nothing to do with a misspelling.

<comment><USER15> The email that was sent to me said that it was because of a misspelled word.

</comment></comment></comment></comment><comment><USER17> I hope this doesn't get down-voted for the confusing title (I know it's the title of the Blog post itself; but out of context it's a bit misleading).

When I moved from Atlanta to Boston, PadMapper was invaluable in finding a place to live. Given that 1) we didn't know the area, and 2) most rentals in Boston and Cambridge are rented by individuals rather than part of larger complexes, it was great to be able to have the listings from Craigslist on a map.

Ditto when I moved from Boston to San Jose. My wife and I were only interested in a house for rent and the vast majority of those could be found on Craigslist. Again, we didn't know the area so having a map was beyond valuable.

I sincerely hope Craigslist does a 180 on this one; PadMapper is a great tool that deserves access to that data.

<comment><USER18> _PadMapper is a great tool that deserves access to that data._

Why do they deserve access?

<comment><USER17> "deserve" is probably the wrong word (forgive me for hastily typing out my comment before running off to a meeting).

My point was really about PadMapper acting in good faith with Craigslist; they do send traffic to Craigslist for the full data on the properties and such. They're a good company doing good things--and it's sad that Craigslist's refusal to innovate has now bled over into "no one else can innovate for us"

<comment><USER18> Ok, no problem. I've just made a habit of calling people out when they use "deserve" a little too frivolously.

</comment></comment></comment></comment><comment><USER19> First it was Carsabi and now Padmapper. While I'm not surprised, as the Craigslist ToS explicitly prohibits crawling their site, it is incredibly frustrating that the website with all the data is unwilling to innovate and unwilling to help others innovate.

Craigslist will continue to be a mediocre, "good enough" solution. Since everyone associates online classifieds with craigslist, none of the other classifieds sites seem to have a chance. It's not even as if services like carsabi or padmapper are competing in any way with them.

<comment><USER20> Craig Newmark has often said that they innovate in response to their community. That would seem to have three effects:

\- Incrementalism: faster horses instead of cars

\- Suburbanism: those in the community want it to stay familiar

\- Majoritarianism: outsiders, potential users, occasional users, mavericks and dreamers have little impact

</comment></comment><comment><USER21> Wow this is really surprising, especially after reading Craig Newmark's answer on Quora just the other day to the question "Why hasn't anyone built any products on top of Craigslist data?". His response:

"Actually, we take issue with only services which consume a lot of bandwidth, it's that simple."

[http://www.quora.com/Why-hasnt-anyone-built-any-products- on-...](http://www.quora.com/Why-hasnt-anyone-built-any-products-on-top-of- Craigslist-data)

<comment><USER22> To clarify for everyone, while the parent read the answer a few days ago, Craig wrote this response in 2010.

<comment><USER23> Good point. What didn't seem a threat then, must now. You should expect some changes in strategy over >1yr.

</comment></comment></comment><comment><USER24> I've used padmapper twice to get an apartment so far, and It's great! I've loved the fact that it's constantly improving and has _exactly_ the features I need: Walkscore, bookmarking, map, proper filters. It's strictly better. To the author: I would have paid for this.

It sucks that this waas mainly an interface to craigslist, and now they're gone. I hope this gets enough critical mass that eventually so that the unusable but unexplicably popular horror that is craigslist apartments just dies an ugly death. Maybe you can even sell it to them, who knows?

All I know is while CL's response is perfectly rational, I can't help but be pissed off that I'm locked into a shitty product that is only surviving because of head start, critical mass, network effects, etc.

Edit: An alternate approach (that would admittedly not help Eric) is to open source padmapper and let people do local installs for themselves. Would be pretty hard to ban that. Although I'm not sure on the legality of scraping.

<comment><USER25> Just a clarification, padmapper will continue to exist with listings from a lot of other places. It will just no longer have Craigslist listings, unless you help send an email to Craig and Jim.

</comment></comment><comment><USER26> Craigslist is in major need of disruption.

Actually apartment finding in general sucks I can't think of a single goto place to look for apartments other then Craigslist and that just proves point #1!

<comment><USER27> I guess a lot depends on where you live - Craigslist in the Bay area is like magic for finding apartments, or a room to rent.

The Bay Area has a very advanced craigslist option, where you can sort not only by geographic area (Peninsula), you can even zoom down to a particular city. Add filtering by price and find a place to live in the Bay area has always (Since 2003) been painless for me - particularly as I don't drive, and so getting a place that is _precisely_ in the right place is important. Housingmaps provides a free map interface for those who find such a thing useful.

I don't actually know how anyone would _improve_ craigslist (for me) - I am not sure what the disruption would be - as the free service provides me 100% of what I would want from such a system.

<comment><USER28> Yeah but that _still_ sucks, because people rarely know which neighborhood they are in. With padmapper, I could see exactly what was where and not have to worry about somebody thinking excelsior is somehow the mission, and the price per room/cats and dogs/walk score/etc... extras are amazing and integrated, especially if you don't know an area.

I guess you haven't really used padmapper if you don't understand how anyone could improve craigslist for you.

<comment><USER29> >> Yeah but that still sucks, because people rarely know which neighborhood they are in.

"rarely" is a tough word. I have found the city to be pretty well demarcated unless you are talking about people confused about minor issues such as what counts as Lower Haight and what counts as Upper Haight.

>> I guess you haven't really used padmapper if you don't understand how anyone could improve craigslist for you.

I used Padmapper as a newbie to the city but quickly found that the latency between postings on CL and the corresponding crawled updates on Padmapper were so high that the postings would have already been hit by a few dozen responses by the time I got it and in a hyper-competitive housing market like SF, this made all the difference.

<comment><USER30> If you're looking on the Peninsula, where school districts and crime levels vary by street within the same time, Padmapper is vastly better than craigslist.

I mean, look at all the misleading fucking tools who post "Palo Alto apartment for $1k/mo" which is actually in EAST PALO ALTO. With padmapper, it is clear. (In Menlo Park, where some areas are unincorporated, some are Menlo Park, some are Menlo Park but a warzone (Belle Haven, aka EPA North), etc., it's even more essential).

</comment></comment></comment></comment></comment><comment><USER31> I expected padmapper to be shut down a long time ago. Mainly because they are an incredible threat to the websites they are scraping all of their data from.

That's probably why padmapper has no way of making money at the moment.

They were hoping they could slink by, stealing other's content, until they were eventually big enough for people to warrant visiting them to list their apartments for rent. And then sell ads.

Don't get me wrong, I love padmapper, but scraping content is scraping content. And it's generally not allowed.

<comment><AUTHOR> You're right, I was hoping I could go on, but I wasn't stealing anyone's content, I was posting links pasted geographically located with stat summaries. Look at a listing on PadMapper - there's no listing text, and when you click through, it goes to the original listing.

<comment><USER31> My apologies. I may have used some improper wording there.

But if I may... you must admit that you have taken "something" from Craigslist.

Whatever you want to call it... be it geographical location points, links-to- listings, content, data, images, etc... it is inevitably some form of information that was originally curated by Craigslist through their 10+ years of existence online and is thus available to be used (and not used) at their discretion.

Canada Post for example charges $5,500/year ([http://www.canadapost.ca/cpo/mc/business/productsservices/ma...](http://www.canadapost.ca/cpo/mc/business/productsservices/mailing/pcdp.jsf)) just to have access to a postal code verification system. It's all publicly available information... if you walked to every city, suburb, nook and cranny in Canada and asked the local people what their postal code is, they would tell you without hesitating.

But if Canada Post gets wind that you may have copied any of that publicly available information from their databases instead of doing the hard work yourself, you can expect a lawsuit at your door fairly quickly: [http://www.cbc.ca/news/business/story/2012/04/13/technology-...](http://www.cbc.ca/news/business/story/2012/04/13/technology- canada-post-postal-code-copyright.html)

I don't think Canada Post had a case in the above example because the company in question didn't actually copy anything from their database, but you did when scraping Craigslist (for however insignificant you found the information to be), see what I'm saying?

Just because you are sending them traffic, doesn't give you the right to take (insert whatever might have been taken here) from them without their permission.

That being said, I love padmapper and have used it numerous times already for myself and friends. I wish you all the best going into the future!

<comment><AUTHOR> Thanks for the explanation! I've talked with lawyers about this in the past week, and facts, totally rewritten summaries of fact, and collections of facts aren't copyrightable in the US, so it's not really an issue of their content, as far as I understand. They do have a right not to be crawled by me, though, especially if it causes a burden on them. I wrote that crawler specifically to be very light on their servers including making tradeoffs to usability (it never recrawls listings), but that's still their decision to make. If there are ways to get the data that don't involve touching their servers, then the TOU no longer applies, if I understand the law, but it would obviously make them angry, and so I'm most likely not going to do that.

<comment><USER32> I agree that this isn't a copyright issue. If you're operating in California, the 9th Circuit is probably your best bet for holding that your scraping of their data doesn't damage or impair their computer system. But given your other comments, it is unlikely you want to raise the funds or spend the time to battle it out in court.

But there is an another way - let us (all of us) get the Craigslist data for you. You could create a browser plugin that would save craigslist posts a user views, and then upload only those posts to a central database that feeds padmapper. With just 1,000 users you would get most but not all the data you currently are scraping.

This is the same approach used by Recap (<https://www.recapthelaw.org/>) as a response to the federal government's paywall to access public court filings.

<comment><AUTHOR> Thanks for the advice! But yeah, court sounds like a pretty big clusterfuck with a potentially bigger clusterfuck at the end of the rainbow. I have a feeling that the plugin would get me sued as well, though I suppose that would be a more winnable battle.

</comment></comment></comment></comment></comment></comment><comment><USER33> Craigslist should buy you, not block you.

I'm basically a Luddite and yours is the one service I always recommend.

<comment><USER34> Maybe they should block you then buy you. Probably would help them in the negotiation...

But in all seriousness, PadMapper was a great service and seemed to do it the "right" way (providing deep links back into Craigslist's site/content), if there is a "right way" to unofficially gather data without use of a public API.

</comment></comment><comment><USER35> PadMapper is like the super helpful feature that craigslist did not figure out how to build correctly, or never bothered to build. I'm disappointed that Craigslist chose to respond by making a great social utility disappear from the market and anger the users. What about charging a license fee? What about acquiring PadMapper? What exactly does Craigslist gain by pulling its index off from PadMapper other than damaging its own public image?

</comment><comment><USER36> Craigslist is an evil monopolist that literally wastes millions of hours of its users time every year through the enforcement of shitty UI (if the amount of money that I save using padmapper is typical). They get a good reputation for some bizarre reason because their CEO is an anti-capitalist, but they are still evil for holding back innovation.

A human lifetime is about 700,000 hours. Craigslist is the moral equivalent of a serial killer.

<comment><USER37> For those who use Chrome web browser, there's some fixes by way of extension that really help save time on CL... My favorite: <http://www.craigslistextension.com/>

</comment></comment><comment><USER38> This is the whole reason I started foreverlist.com. Besides better/more photos, geolocation, comments, etc..., I always wanted to open up the api and let anyone develop on top of it. Alas, competing with craigslist is folly. I regret cloning the design of CL, but the logic was that it would be simple for people to use.

I absolutely LOVE pad mapper and hate craigslist even more now. Many good sites have been killed off by craigslist.

[edit spelling]

</comment><comment><USER28> So should they ban Google? Because I use google to search for things on craigslist with much more success than craigslist's system, in the same way I used Padmapper to find a room/house (3 times in the past year... I move a lot)

This is stupid.

</comment><comment><USER39> Carsabi (YC W12) was also affected by this recently. Especially annoying midway through my car search. It is really disappointing that Craiglist won't license their data.

<comment><USER40> I would say this pretty much killed Carsabi.

<comment><USER41> Actually not. Enough of their content is direct from dealers that it looks like they will be able to bounce back.

<comment><USER39> Yeah, they still have quite a lot of content, and to be honest craigslist listings were some of the worst on there. Still, it was handy to just have _one_ go to place for car shopping.

</comment></comment></comment></comment><comment><USER42> Kudos to CL for standing up for user privacy. When I use CL it's with consideration of their TOS which bans these practices.

When padmapper copies the data they violate my privacy, my copyright, and the agreement between myself and craigslist as outlined in the TOS. What padmapper is doing is a violation of many data privacy laws like PIPEDA, EUDPD, and various copyright acts.

If padmapper wants the data why don't they just obtain consent from the CL posters, instead of copying it without consent.

Would the padmapper team be ok with me deciding to stick their logo where ever I deem it necessary? I bet they'd probably sue me for copyright and trademark violation.

<comment><USER43> This isn't making a permanent record of casual encounters.

It's putting apartment listings on a map. People listing apartments tend to want it to be widely known.

<comment><USER44> People listing apartments tend to want it to be widely "known."

those same people are already getting their listing widely known, by posting it on CL.

</comment></comment></comment><comment><USER45> Here's the email I sent to Craigslist. I tried to use the following template: intro (what are we talking about), how padmapper is good for craigslist, why I care about padmapper, course of action.

Feel free to reuse it.

\----

Hi Jim and Craig,

I learned today that Craigslist has sent a Cease and Desist to padmapper.com. Padmapper provides a service to visualize on a map rental listings from sources like Craigslist.

Real estate is all about location, and let's be frank, Craigslist's location- based search sucks. Padmapper helps the searcher easily see where real estate is located. It turns a grueling search into a breeze. It adds great value to Craigslist's listings without competing with Craigslist's business.

Padmapper, along with Craigslist, were essential in helping me find my current home in the San Francisco Bay Area. Without Padmapper, I wouldn't have used Craigslist in my search.

I'm sure you can find a mutually beneficial deal with Padmapper, instead of shutting them out entirely.

</comment><comment><USER46> Just out of curiosity, why would they not want you to use their classifieds? Unless I am missing something, they don't collect revenue for listings. I thought Craigslist was all about spreading good will. If Pad Mapper provides a more efficient way of spreading that good will, then why not?

As a user of PadMapper, I am very disappointed to see this happen. Craigslist is almost unusable if you don't know the area you are looking to move to.

<comment><USER47> As best I can tell, it's a matter of corporate philosophy. Craigslist is predicated on local, person-to-person interactions. It has long taken pains to curtail uses of the system which allow aggregation of multiple CL local sites (there is, for example, no convenient way to search all of California for listings), or to include CL listings in third-party services.

As a long-time user of Craigslist, and a recent fan of padmapper, I'm disappointed, to say the least.

I do suspect it's time for some disruption in CL's space.

</comment></comment><comment><USER48> I'm surprised how many people here are defending Craigslist. They've got a history of both providing a terrible experience for apartment seekers and shutting down any attempts to use their data to provide a better experience. That would be okay in a competitive market, but they've got a strong monopoly in a market with a high barrier to entry. They're playing the textbook role of an evil monopolist.

<comment><USER49> Craigslist is a company whose terms of service were violated by PadMapper. Whether or not PadMapper provides a better experience is irrelevant: Craigslist has a right to defend its content and lists by whatever means it sees fit. Any businessperson on this site would expect to have the same rights.

Craigslist is not in the business of helping anyone but Craigslist be successful. It's not monopoly; it's business.

<comment><USER48> Companies are not free to do whatever they want. The government compels monopolists to avoid anticompetitive behavior, and even breaks companies up, for examples. The market in this case is a little hard to define, since apartment seekers aren't purchasing the product, so I'm sure it's not actually illegal, but it sure seems to violate the spirit of these laws, which are designed to protect consumers from exactly this kind of behavior.

</comment></comment></comment><comment><USER50> This is horrible. I used Padmapper extensively when looking for an apartment here in New York two months ago. I guess I'm never moving again...

</comment><comment><USER51> Padmapper has been a wonderful service. It is confusing why Craigslist is blocking it unless they (Craigslist) are thinking about coming up with similar user interface soon. Craigslist has been hiring UI designers.

</comment><comment><USER30> Why not buy a license and go hard on mobile? Then use that to get more direct listings, ideally with ACTUAL STRUCTURED DATA unlike the shit that is craigslist (i.e. closer to MLS, or even better, closer to the proprietary systems used inside large real estate investment companies), and then re- release a web version using only those listings.

Maybe focus on SFBA only, since the real estate situation is acutely broken here AND smart+rich people still rent here (unlike Texas or the midwest, with NYC being its own weird market).

<comment><AUTHOR> Maybe, if you want to chat more about this, email me at firstname.lastname@example.com

</comment></comment><comment><USER52> At the end of the day it is Craigslist's data. I know padmapper was useful for a lot of people so the next smart move is to build your own data. A "craiglist" focus on rental only with the same features. The sad reality is you need your own platform to scale any type of services like that.Is it going to be easy and fast to build. The answer is no. However the reward and opportunity are huge....

Of course it would be great if craigslist could have an API or licence his data but it won't happen in a bear future ...

<comment><USER53> _At the end of the day it is Craigslist's data._

Nope. It's the users' data. Craigslist only has a license to use it.

<comment><USER54> In that case Craigslist is doing right by its users, the licencor of the data, since the licensor has not granted a license to padmapper to use said data.

</comment></comment></comment><comment><USER55> I think its pretty well documented that craigslist is known to shut down services that use craigslist data. I really don't think should come as a surprise to the creator.

</comment><comment><USER56> Unfortunately, scraping another site's information is generally not a long- term viable business plan (speaking from personal experience). I suppose CraigsList, much like insurance companies, doesn't like its valuable information being displayed on other websites without endorsement or consent.

That said, I love PadMapper, having used it a couple of times for finding apartments in the Bay Area. So it's a bummer that they won't be able to show CL listings anymore.

</comment><comment><USER57> Good luck, I think it's a sign of your success that this has happened. Providing more value with their data than they do is a threat to their business.

<comment><USER58> I can't see Craigslist ever stepping up their game and providing meaningful visualization. I can't really see why CL cares about where people are reading the listings - their business model is based on selling the listings, not monetizing the viewing of them. I suspect it's out-of-control lawyers at work.

<comment><USER59> Craigslist is closer to a non-profit than a business. You can't model them as a self-interested agent.

<comment><USER60> And like many other non-profits, the quality of the service shows - <USER3> service is merely "good enough" (in many categories, far less than good enough), with no competitive pressure to improve.

We have already seen the meteoric rise of many niche, category-specific Craigslist competitors eclipse CL in scale and influence, particularly in real estate. I suspect due to Craiglist's general crappy experience this fragmentation will continue.

And I for one think it's a good thing.

<comment><USER59> Well, the lack of competitive pressure stems more from network effects than being non-profit per se.

Because of this, I don't see the fragmentation as a good thing. One of two things will happen in each craigslist market (rentals, used goods, ridesharing, etc.): either a single for-profit company will ride networks effects and establish a monopoly, or there will be severe fragmentation.

If the former, then you can expect some features to improve (in order to steal the market from craigslist in the first place) but many other things to languish due to a lack of competitive pressure. (Also, if it's a for-profit company, you'll pay money.) This is ebay.

If the latter, then it will no longer be possible to go so one site to search in that market, greatly decreasing consumer utility. This is how classified ads worked before craigslist.

Much better is a well-run, benevolent non-profit. Wikipedia, though not flawless, is pretty much the epitome of this idea. It's scary to think what it would be like if Wikipedia's data was spread over dozens of websites of varying quality and motivation. (Then again, in the scenario no one would bothered to write most of the articles anyway.)

Craigslist doesn't fit this exactly because they impose a philosophy ("localism"?) which most people don't share and which decreases usability.

<comment><USER60> There are, for example, real estate websites that are eating the CL for-rent category's lunch right now. Craigslist isn't doing much in response to these competitors, despite the fact that in some markets (particularly SF and NYC where I have experience) Craigslist is now very firmly in second place (or worse).

I've been apartment hunting in NYC lately, and it's mind-boggling what proportion of Craigslist posts are scams compared to other sites like StreetEasy or NYBits.

IMO if Craigslist was run more like a scrappy startup looking for their big exit, they'd have more than responded to these competitors by now. The network effect exists, but IMO is an insufficient explanation of their lack of action.

I agree that the fragmentation is, all things being equal, a bad thing for users - but considering how utterly useless Craigslist's moderation systems are (many categories are full of duplicate posts, spammers, and scammers), and how poor the UX is otherwise (search? hah!), I welcome it. Fragmented sites like PadMapper, StreetEasy, etc, have done _way_ more to ease the life of renters than Craigslist in the last decade.

</comment></comment></comment></comment></comment></comment><comment><USER61> This is a damn shame. I used padmapper to find my last two places. CL should've been doing this feature on their own, navigating rental listings on their site is a nightmare. The map feature was a godsend.

</comment><comment><USER62> Curious if Live Lovely (<http://livelovely.com/>) will get a similar cease and desist. They do essentially the exact same thing, but with a nicer design.

<comment><USER63> Except Padmapper works everywhere craigslist does. I went to livelovely just now and they don't have my city. It's not exactly a small town. Also their filtering is nothing compared to padmapper.

On a more related note, I would imagine any sites like this that come up on Craigslist's radar will get a similar notice.

</comment></comment><comment><USER64> Disappointing. I'm currently living in an apartment I found via Padmapper and Craigslist. Padmapper was absolutely crucial, without it I wouldn't have found this place.

I understand why they're complying with the C+D, but I wish they'd fight it. Craigslist is better than printed classifieds, but that's about the best thing you can say about it.

<comment><USER25> I think it's harder for small guys like Padmapper to fight them, because 1) There's no money for lawyering 2) Craigslist says that they listen more to users for UI.

So if you'd like to see CL give Padmapper license to use its data, send them an email. Your voice as a CL user weight more than Padmapper's would.

</comment></comment><comment><USER65> Craigslist does not allow scraping. Beg for help and for support for breaking their rules elsewhere. This is Hacker News, not "help me retain a business model after I broke rules" central. You could alternatively hire people to spam them with nice requests to reconsider.

</comment><comment><USER66> Just sent a mail to {craig,email@example.com. Not sure if it will help but if they hear enough noise about it, maybe they'll change their stance.

<comment><USER67> It's most likely an indication that they'll be developing their own UI.

</comment></comment><comment><USER68> Could Padmapper reform itself as an extremely intrusive browser extension?

Is there any evidence CL was pressured by any privacy groups or regulators?

</comment><comment><USER69> The pro CL comments here are ridiculous and so anti-innovation. "Don't crawl my data bro!". PM wasn't modifying the data in any way, just redis playing it on a damn map. Now CL is useless, yet again, for housing.

</comment><comment><USER70> I wonder why CL doesn't create an API and charge for it's usage. The data is clearly valuable and they own it, but that doesn't mean that it can't use multiple views.

<comment><USER71> This is only useful if the teams scraping CL have budget to pay a market rate for the data. Considering that CL seems to have a lock on this data for many markets, I'm guessing that's worth a lot of money (which startups will not be able to afford).

<comment><USER72> If they cannot afford, then they should not be there. Scraping illegally is not a long term solution. Craigslist should conduct some market research from their end to charge a "reasonable" fee which fills in their coffers as well as allow development an ecosystem around their data. Even DuckDuckGo pays for Yahoo! BOSS Search API to show their results. Not sure why CL hasn't thought about it.

</comment></comment></comment><comment><USER73> I'm pretty sure that Craigslist disruption is coming, and it's going to be AirBnB.

I'm not too worried. What craigslist is doing (ie, sucking) isn't sustainable.

<comment><USER74> Hah, people have been saying that for a decade. What people fail to realize is that Craigslist is ultra successful _because_ of what it does, not _despite_.

</comment></comment><comment><USER67> Why the hate on Craigslist? It's their information, and are free to do what they want with it - most likely, develop their own platform.

</comment><comment><USER75> Quite a misleading title..I think many clicked thinking to read about an unexpected shutdown of Craigslist actually

<comment><AUTHOR> Sorry, I didn't mean it that way - if you know what PadMapper is and see the URL, I think it makes sense.

<comment><USER76> With the OP here, I saw the thread title and silently went "thank god" to myself immediately knowing what this meant, having used PadMapper _very_ extensively in the last year after moving to Austin. I knew it meant a good portion of ads being gone, but I was going to get much more _qualitative_ results in finding a condo.

I really hope PadMapper can make up the volume but continue to have good results for people searching. It's a wonderful resource.

<comment><AUTHOR> Haha you could always filter out Craigslist with the sources filter (though it's a bit hidden)

</comment></comment></comment></comment><comment><USER74> I propose that Craigslist has been ultra successful _because_ of its actions, not _despite_. I think it's success is due both directly to how it has built and operated the site as well as due to the overall mentality that fends off all the lame ideas from the armchair product designers.

</comment><comment><USER77> "They allow mobile apps to display their listings if you buy a license from them, but not websites."

Really? They used to allow any website to use their data for free as long as they didn't run ads against it or monetize it in any way. Kexter.com has been doing exactly what you're doing for almost ten years now.

</comment><comment><USER78> Padmapper's a brilliant solution. I'd like to think that Craigslist is building their own great solution, but knowing how slowly CL moves on everything, I sincerely doubt they'd build a worthwhile replacement. Bummer!

Padmapper's helped me and a few friends find our apartments in SF.

</comment><comment><USER79> Could it not mean that they are coming up with some padmapper like functionality? Padmapper of course is the only usable rental search site after craigslist. The rest of them like rent.com and apartments.com are absolute crap. I hope they work something out.

</comment><comment><USER80> How is this different from when Bing was using Google search results? If people want to use Padmapper great. if people want to use Craigslist then great.. Am I missing something here? I don't see the problem. What if it were the other way around?

</comment><comment><USER81> This sucks. Craigslist is not setup to handle apartment searches at all. I literally just finished finding an apartment and I solely used padmapper. I can't imagine having to go through the process just using craiglist search. Bummer.

</comment><comment><USER82> Cached version: [http://webcache.googleusercontent.com/search?q=cache:http://...](http://webcache.googleusercontent.com/search?q=cache:http://blog.padmapper.com/2012/06/22/bye- bye-craigslist/)

</comment><comment><USER83> I personally would have tried to find a way to work with them. But it's not my site.

Craig isn't doing anything wrong or unethical here. Quite the opposite; Craigslist continues to deliver _exactly_ what its users signed up for.

</comment><comment><USER84> I'm sad to hear this. I used padmapper to find my place in NYC among an incredibly scam-filled and otherwise difficult to navigate sea of lies and frauds. PM helped me utilize CL in a way their filters could not.

</comment><comment><USER85> I used PadMapper with great success numerous times, I hope it can survive this.

</comment><comment><USER86> Once read about how its a bad idea to base the core of your business around someone else's platform. Once they(and their lawyers) decide to re-evaluate your relationship with them it can nearly destroy what you've built. Testament to this are the number of apps based on twitter that died when twitter decided to up their offering and cover what they were doing. How to do it well is Zynga's relationship with Facebook

The people at PadMapper are doing a good thing looking for other sources but like the author said Craiglist was/is an important source of pad listings.

All the best to them

</comment><comment><USER87> I decided to consider moving to the US on a whim about 6 months back (in the end I didn't) and decided to try and find an apartment and guage the cost of living. People recommended craigslist as apparently the US doesn't have a rightmove (<http://www.rightmove.co.uk/>) type site. Craigslist was probably the worst experience I've had trying to find an apartment... I can't believe that people actually use it, why wouldn't they build a padmapper esque system (or acquire padmapper)?

<comment><USER88> Craigslist is, unfortunately, the "best" example of the network problem. Due to its popularity, it is entrenched.

That said, there are methods of disruption. AirBnB bootstrapped off Craigslist to get their initial dataset, and have since moved to their own content. That's probably the strongest approach if you're lucky enough to be able to move quickly when you need to.

<comment><USER89> The thing is, I think Craigslist's network effect is exaggerated. Its network is very clustered because nearly all of it (particularly housing) is focused on people in a particular area. Compare to Facebook, which has a large degree of inter-connectedness between all clusters.

If there were a site that had, say, all the rentals available in the SF Bay Area, it would be highly useful to you, as it would have gobbled up most of the benefits of the network effect available (of course, outsiders moving in wouldn't automatically know its name like they do for Craigslist, but that's relatively minor).

Once you've reduced your problem to "develop a really strong network in a very localized geographic area," it seems to me like the solution would become in reach, with a lot of hard work. You could even scrape pretty heavily early on, or enter it by hand if somehow that got around the ToS.

Maybe start at Berkeley or similar college town, maybe reducing yourself to using physical paper and highly targeted online advertising, and rely on students heavily to get the word out. Make it ridiculously easy for someone to post an ad on your site and have it automatically propagate to CL. Gradually expand to different parts of the Bay Area, and then pick out other localities to target. NYC, maybe, especially if you can somehow make its real estate market less dysfunctional.

</comment></comment></comment><comment><USER90> Craigslist needs to be stopped. This is the third incredibly useful service I've seen shutdown in as many weeks. I was heartbroken when Carsabi had to stop indexing Craigslist a couple weeks ago.

</comment><comment><USER91> Could the C&D be PadMapper violating the use of the trade mark?

<http://www.padlister.com/listings/new>

"Better looking, effortless Craigslist Ads."

</comment><comment><USER92> Was there some attempt made at monetizing Padmapper? IIRC Craigslist isn't super hard core about not-for-profit projects using their data, but takes a hard line on commercial use.

</comment><comment><USER93> Let me guess your young and your pathetic education taught you very little, and your sheltered life made you think everything would be handed you on a silver platter.

Instead of whining, you either need to offer large sums of cash to CL, or some deal. CL does not owe you a living. Your crappy little app is just skimming the cream off the top of CL's infrastructure and you expect CL to give you a free ride? Not going to happen. You are simply lucky you got away with it as long as you did.

</comment><comment><USER94> FWIW Craigslist is a ghost town in Australia and virtually useless for buying & selling the type of stuff I used it for frequently when I lived in California. For example here's the furniture for sale section for Melbourne: <http://melbourne.craigslist.com.au/fua/>. Seems that in Australia people use Gumtree.com.au or even ebay.com.au for online classifieds.

<comment><USER95> gumtree is where it's at for apartment rentals and other big hard-to-ship odds and ends. then there's carsales.com.au for automobiles, and ebay for everything else. seems to work out okay.

</comment></comment><comment><USER96> Really sorry to hear this. I've successfully found 3 different apartments in perfect locations through PadMapper (and one that I'm moving into in 2 weeks actually). And I've had several friends use it too.

Great job with all the time you've put into the site, Eric - hope the service pulls through this somehow, but also looking fwd to seeing what you work on next if you decide to move on.

</comment><comment><USER97> Solution/Question:

Could the padmapper guy just make an application, which he sells for $10-$15, with the same functionality, except all the "scraping" happens from the user's computer or browser?

And if this violates the TOS, assuming I can sell a browser which only visits Craigslist -- then where is the line where a browser does too much "formatting" and becomes a TOS violation?

</comment><comment><USER98> Sometimes people seem to forget that the most frequent users of Craigslist, and the ones that keep it popular, are not complaining about it having bad UX. We are.

I want PadMapper to succeed (it helped me find a place super-quick for grad school in the fall), but Craigslist has every right to legislate their content. We'll see what they do/don't do about it.

</comment><comment><USER72> If I were at Craigslist, I would strongly look at commercializing their listings. By that I mean charging a fixed amount (say $5) for every 1000 results sent to the requester (such as Padmapper). That ways Craigslist gets to keep the revenue while other companies can innovate on top of the massive data which Craigslist has.

</comment><comment><USER99> This post could use a new title, unless you want to scare HN'ers into thinking Craigslist is shutting down.

</comment><comment><USER100> CL is obviously in their full rights to do so, but I really wish they could have waited a couple weeks before they did it. I'm moving to another state this Tuesday and PM is especially helpful when you don't know the neighborhoods.

</comment><comment><USER101> as someone typing this in the apartment they found via a Craigslist ad on PadMapper (and as someone who has recommended it to several people because of the feature). time for a mobile PadMapper with a proper license for the data? :/

</comment><comment><USER102> Craigslist is hiring Senior UI / Usability / Front End Engineers - maybe they plan on releasing a similar feature?

<http://sfbay.craigslist.org/sfc/sof/2841395914.html>

</comment><comment><USER103> I would make the same argument that many are making with Cragislist about IRC, the use of multi-digit telephone numbers and a huge variety of other entrenched technologies that work reliably for enormous numbers of people.

The entrenched technologies are clearly excellent in the context of what they do. Can additions be made which allow one to comprehend data faster, summarize and/or visualize groups of data, filter out garbage and/or present things in a simpler way? Of course.

The agreement-violating leeching activity of some can be easily be forgiven because is not happening to your data, your servers, your love and your life's mission.

In my mind, remoras like this are exposing themselves to additional risk by doing something that is in violation of TOU and potentially a violation of various laws.

There are so many fitting cliches here that it's making my head spin.

<comment><USER104> Or maybe those that are "forgiving" find that this "leeching" is creating a positive benefit for society as a whole, and that ultimately, IP laws are there to benefit the public.

</comment></comment><comment><USER105> Craigslist is one of the greatest blown opportunities in the history of business.

</comment><comment><USER106> Could Craiglist shut down every single IP who scrapes? Could you build Padmapper to be less 'realtime' and more 'tell me the city and I'll set up a client-side scraper, come back in 15 minutes?'

</comment><comment><USER107> I found this : [http://m.techcrunch.com/2012/06/22/padmapper-craigslist- ceas...](http://m.techcrunch.com/2012/06/22/padmapper-craigslist-cease-and- desist/)

So no it was not bye bye.

</comment><comment><USER108> >> They allow mobile apps to display their listings if you buy a license from them, but not websites.

PadMapper already has an app so couldn't they continue to use the listings in the app?

</comment><comment><USER109> Bummer. I'm completely mystified that CraigsList doesn't have an API. Fortunately, others do: <http://3taps.com/> [no affiliation]

</comment><comment><USER110> Can you open source padmapper so we can run it ourselves? That way, no matter what Craigslist does, people would still be able to use the great service.

<comment><AUTHOR> Hm, that's kind of an interesting idea. I'd be pretty embarrassed at all the hacks I've added over the years, but I'll definitely keep that in mind.

<comment><USER111> Yes. Please open source it...

</comment></comment></comment><comment><USER112> I don't think I would have ever found a place to stay in SF after moving here 4 months ago if it wasn't for CL's data and PM's ease of use.

</comment><comment><USER113> This is an existential threat to Padmapper. Which is very sad to me, since it's easily one of my favorite sites to search for anything.

</comment><comment><USER114> This really sucks. Finding an apartment in Boston on Craigslist is nearly impossible without PadMapper.

</comment><comment><USER115> Here's an idea-- why doesn't Craigslist just buy padmapper? They certainly have enough cash to do so.

</comment><comment><USER116> Nooooo!

I was going to start looking at places in Padmapper next month, for my upcoming move!

Is there any other similar service?

</comment><comment><USER117> How about all the ipad apps that access craigslist data? how do they get away with that???

</comment><comment><USER118> Counting down for a new wave of PadMapper clones / CL scrapers to start popping up.

</comment><comment><USER119> oh sad to hear this. padmapper service with craigslist really helped me!

</comment><comment><USER93> Just read the entire thread, got to be one of the most sock-puppet, straw- manned, astro-turfed, meat-puppeted mess of self serving wanna be entreprenuer whining ever.

CL is successful, get over it.

<comment><USER120> You're absolutely correct.

</comment></comment><comment><USER121> Can you please open source all the code specific to craigslist data extraction so that others may incorporate your work into open-source projects?

</comment><comment><USER122> Take them to court. I will donate to your legal fund.

<comment><USER53> On what grounds, exactly? They have every right to deny you access to theirs servers.

<comment><USER122> They have every right to implement technical barriers to scraping. Whether they can invoke the law to establish a legal barrier to re-publishing their proprietary-but-public facts -- not even the apartment summary text, just extracted metadata -- is not at all clear. When an auction aggregator was sued by eBay for doing something similar, the judgment in eBay's favor rested on the fact that the aggregator was posing a real undue technical burden, not the inherent fact that they were scraping.[1] Since I assume Padmapper does intelligent scraping, they might have a fairly good case.

[1]: <http://en.wikipedia.org/wiki/EBay_v._Bidder%27s_Edge>

<comment><USER123> I think you may be confusing two issues.

It's almost certainly true that they could republish the _data_ without violating any law. That has nothing to do with whether they have the right to scrape the data from the website.

IE do not confuse their right to republish the data with the right to get it from a particular source in a particular way.

The line of cases you are citing is also about trespass to chattels, which is a pretty bad claim to make about website scraping (for the reasons the california supreme court set out in Intel vs Hamidi). It was mainly used back in the days when courts barely knew what websites were. Trespass to chattels may still get used when you can show actual harm caused by the scraping, but most claims nowadays would be around terms of use violations, which are a much more grey area (some courts have gone for it, some not).

</comment></comment></comment></comment><comment><USER124> Hi Craig and Jim,

I was disappointed to learn you dont want PadMapper to use your listings. Their site and Carsabi are two sites I recently used to find my apartment and car, respectively, only after I had given up on dealing with your firehose and sparse search options. In using both sites I knew full well I was simply using them as a better, more elegant, more feature rich version of CL. Both ultimately referred me right back to your site for specific listings, which was exactly what I wanted them to do.

I dig and respect that you guys are staying true to your mission, running a lean company and keeping it simple. I've read all I can find about both of you and how you've built and how you run CL. You are two of my heroes. But you are really betraying your mission and your credibility with this move. You should let your data be completely open. Let people do whatever they want with it. CL is an institution. You are a backbone of the internet. Just like Facebook and how they share their graph with anybody who wants to use it, as it spreads the FB brand everywhere - you should embrace people wanting to use your data. It will only serve to spread the CL brand further and even more important, it will make the lives of people all over the World better. I thought that was what you stood for. I hope you don't prove me wrong.

Regards,

-Bill

</comment><comment><USER125> I get that CL has a fast UI that is very easy to understand for non-technical users. But why can't they somehow work with others who want to improve the experience?

There have been several sites that I have personally used to aggregate searches, eliminate duplicates, filter on different attributes, etc. that all were better for me than CL. (Craiglook, padmapper, etc.)

Every, time, CL flips them off and threatens suit.

FFS, guys, just strike a deal with some of these devs and make a 'tech- friendly' portal for people who want actual search capability from your site.

</comment><comment><USER126> Total BS Craig... You should be REWARDING innovation, not joining the cadre of douchebag trolls that try to kill it.

</comment><comment><USER127> You already got the traction. Offer something complete now. It's already better.</comment>


"Life is short. No recession warrants wasting precious time." - <AUTHOR> http://goodexperience.com/2009/04/on-hospitality-in-a-t.php

<comment><AUTHOR> This is from Danny Meyer, whose Shake Shack has been the planned location of a few NYYC meetups, and the actual location of at least one.</comment>


Visualizing the Similarity of Human and Chimp DNA - <AUTHOR> http://www.cs.ucr.edu/~eamonn/DNA/ <comment><USER2> On a second thought, it seems it could be used for 3D shapes in genetic algorithms, with each English letter representing one neighboring voxel. If it hods true that similar DNA results in a similar shape, it could work very well.

</comment><comment><USER3> Seems like there are much more intuitive ways to show these differences, something like the plot on Dr. Moran's blog:

[http://sandwalk.blogspot.com/2012/01/whats-difference- betwee...](http://sandwalk.blogspot.com/2012/01/whats-difference-between- human-and.html)

Although the 3D plots are neat, they kind of feel like 3D barcharts: they add very little over a more straightforward representation (like a heatmap instead).

</comment><comment><USER4> Gave it a shot in HTLM5/WebGL. Check it out:

[http://grondilu.github.io/dna.html](http://grondilu.github.io/dna.html)

</comment><comment><USER5> Can someone clarify this for me? From watching their video, it seems like all they've done is build a tool that visualizes DNA? I don't understand how the tool assists with visual differences in DNA. I expected maybe a single DNA sequence in one color with the differences highlighted in a contrasting color. Maybe the video just didn't do a good job of explaining things.

</comment><comment><USER6> It doesnt seem like a lot of thought has gone into making the colour / drawing choice actually visually significant. Eg. given blue is much harder to see here than red, are we over-estimating similarity because there is a lot of red?

I suspect they need to talk to some visualization & human vision people to make this compelling.

<comment><USER7> Like Pepe said below, each letter spawns in a specific diredtion, so A, or red, always goes up, and you can check that in the beginning, where there are only few lines. so the final shape is more important than color. the color is basically irrelevant, except for the fact that maybe you can see which of A, C, G, T appears more times in that dna sequence.

</comment></comment><comment><USER4> Neat idea. Someone needs to implement that in HTLM5/Webgl.

Couldn't resist trying something similar with 2D canvas:

[https://output.jsbin.com/zorepe](https://output.jsbin.com/zorepe)

It will definitely look better in genuine 3D, but that's a bit more painful. Totally doable, though.

</comment><comment><USER8> The page was last modified on 27th June 2011, as displayed by Firefox page info dialog.

There might be more recent approaches like the one mentioned by aaronmck which is from 2012.

</comment><comment><USER2> Sorry for a bit off topic coment, but it made me realize that Gattaca consists only of DNA "letters".</comment>


Humans Migrated to Mongolia Much Earlier Than Previously Believed - <AUTHOR> https://www.ucdavis.edu/news/humans-migrated-mongolia-much-earlier-previously-believed <comment><USER2> I love how the Denisovans were named after a guy named Denis who lived in a cave. Very Adamsist.

The time that humans arrived keeps getting pushed back as older artefacts are found - can we plot dates against discovery time to estimate when it asymptotes? (And when we will get there, within some ε.)

The date for Aboriginal Australians is 65,000 ATM. Mongolia seems a similar distance. [https://wikipedia.org/wiki/Aboriginal_Australians#cite_note-...](https://wikipedia.org/wiki/Aboriginal_Australians#cite_note- ClarksonJacobs2017-6)

<comment><USER3> I've gotten to the point where I suspect the "beginning of humanity" is actually the "first slot in the loop buffer of history" bounded by the erosion of the surface of the Earth.

Even stone tools only seem primitive because the plant and animal materials that completed them are gone, unlike metal tools which can be made of 100% durable materials.

My default assumption is that 50k years ago there were human civilizations every bit as rich and comfortable as ours, if not moreso on both counts. And that if you were transported there and taken in by a family you would feel quite at home.

I have never seen any material evidence to the contrary. History for political reasons has discounted the possibility of such civilizations but after many years I finally realized "we have existed roughly the same way for a hundred thousand years or more" is just the null hypothesis.

<comment><USER2> The downvoters don't realize that behaviourly modern humans seemed to emerge about 50k years ago ([https://wikipedia.org/wiki/Behavioral_modernity](https://wikipedia.org/wiki/Behavioral_modernity)), dated by cave art. In a fundamental sense, those people were as smart as us. What did they do with their time?

</comment></comment></comment><comment><USER4> I one took a trip to Mongolia in the dead of winter one year. -45 C at night, -25 C in the daytime. I hate cold, but I wanted to see life in such an extreme environment. Very eye-opening, but I was glad I had my winter hiking gear and ice climbing socks with me just to walk around town. I can't imagine how it was for people thousands of years ago.

A few photos here: [http://dheera.net/photos/places/mongoliawinter](http://dheera.net/photos/places/mongoliawinter)

<comment><USER5> There's the possibility that the humans in that area at that time were migratory, i.e. only visited it in the summer.

<comment><USER6> Or that the local Mongolian climate was more mild at the time.

</comment></comment></comment><comment><USER7> Are there any studies that check % Denisovian DNA versus living elevation? All I hear about Denisovians are that they were more adapted to higher elevation living. So if there were a mixing event or multiple mixing events however many tens of thousands of years ago - could it possibly be that the offspring naturally end up at different elevations based on what was optimal for their genetics?</comment>


Why European Animosity to Facebook Won't Halt Libra - <AUTHOR> https://decrypt.co/9376/europe-facebook-libra-cryptocurrency-regulation-cardano <comment><USER2> The article doesn't explain why. The word "why" is only in the title.

The article does a better job explaining who Hoskinson is than explaining what it promises in the title.</comment>


Magic Trumps Math at Web Start-Ups (NYT) - <AUTHOR> http://dealbook.nytimes.com/2011/06/17/abracadabra-for-internet-start-ups-magic-trumps-math/

<comment><USER2> >LinkedIn generated $3.4 million in profit last year. Using adjusted earnings, which accounts for items like stock-based compensation, it reported nearly $48 million.

I think this quote nails the difference between this bubble, and that of 2000: A decade ago, companies were going public with absolutely no profits (or even revenue). This time, some companies are actually making a bit of money, but it's over estimated and somehow being used to justify multi-billion dollar valuations.

</comment><comment><USER3> Obviously GAAP rules are not the end all and be all of reality but this article seriously made me step back and marvel at the audacity of some of the companies. Loose accounting seems to present before every bubble stretching back to the East India Company.

We are certainly headed for a correction of some kind.

</comment><comment><USER4> If you're interested more on how companies previously invented metrics to make themselves look more valuable than they are and how this led to speculative insanity, and a general introduction to investing and economics, try this book:

[http://www.amazon.com/Random-Walk-Down-Wall- Street/dp/039332...](http://www.amazon.com/Random-Walk-Down-Wall- Street/dp/0393325350/)</comment>


Ticketstumbler (YC summer 08) "growing 50 percent a month" - <AUTHOR> http://www.masshightech.com/stories/2008/11/10/weekly15-Software-startups-are-launching-with-just-bootstrap-funds.html

<comment><USER2> I actually didn't want this article submitted so I flagged it. Kidding, kidding. There are, however, some out of context quotes and a ridiculous looking picture (the photographer was nice, but quirky). Oh well, I <3 press.

 _To clarify:_

-We've never said that we'll never take money. In an ideal situation, startups would not have to take money and if we can get away with it great, if not that's okay. We're well aware that the odds are against us on this one.

-We've been very open with this with our potential investors. The sharp money is still investing in this economy and luckily some of them are still interested in us. You think a VC will argue with you when you say you want more months of operational information so you can make a better informed, data-driven decision? Nada. I've talked to four potential investors this week alone.

-Due to the economy, sales have stayed relatively flat as average sales have gone down while volume has gone up. However, we're still working out bugs and have over $1 billion worth of tickets on the site. Most people don't even know we have concert tickets yet, which is by design. We'll be chasing more PR soon, which will hopefully lead to real salaries for myself and Tom.

-We don't have a model, a deck, a pitch book or executive summary. We'll make them on demand if they are ever needed. If you want to do it right, it's a very time consuming process. In the short term we thought it was a better use of our time to talk to our users (I respond to literally every non-form letter email), create content, develop and sell.

-Obviously models can reflect growth rates. My point was: when you are growing quickly in the early stages, by the time you get to the final boardroom meeting, everything can very easily be out of date. This is especially true when you haven't spent any money on advertising.

-Month over month traffic is up 67% as of today. This is largely due to Obama Tickets, more pages being indexed, SEO and good ol Google.

-I haven't had a haircut since I quit my job. \m/

Edit: Of course as I write this our contact page is down. We're on it.

Edit 2: It's back up.

<comment><USER3> Willikers, Dan, you're like the uber-caveman in this picture!

<comment><USER2> PG prefers to call us "barbarians".

</comment></comment></comment><comment><USER4> I have to say, Congratulations to Tom and Dan. Turning a profit from the get- go is very rare, and they managed it quite well. It's great to see that snowball rolling downhill...

Cheers!

<comment><USER2> Thanks.

Things are/were pretty close though. If we would have moved to SF, bye bye profitability. That's one of the main reasons we're in Boston.

</comment></comment><comment><USER5> I think Ticketstumbler's unofficial motto is "Love us with money". Buy tickets y'all

<comment><USER2> Now with concert ticket goodness. Hopefully theater tickets, more providers and roughly $2 billion worth of total tickets in early 2009.

<comment><USER6> Hmm... It's probably prudent to reach $2 million before talking about $2 billion.

Wait, are you saying you'll actually sell $2 billion worth of tickets, or just offer them for sale?

<comment><USER2> lol...offer them for sale. That's how much inventory we hope to have. Currently there are $1 billion worth of tickets offered for sale on TicketStumbler (we don't own the tickets, just like Expedia doesn't own the air planes).

Shit man, if we had $2 million in sales already I'd be on an island somewhere. It'd be 99% profit.

<comment><USER6> Right, yeah. It just doesn't seem like Ticketstumbler will necessarily become more valuable just by increasing inventory. (So why use that fact as a bullet point when describing why Ticketstumbler will become valuable?) Increasing inventory is necessary but not sufficient.

Your best bet is probably to focus on ways to differentiate Ticketstumbler. For example, what reasons are there for me to buy a ticket through Ticketstumbler than from any other source? Etc.

<comment><USER2> It's valuable because the more ticket providers we have, the cheaper prices will be and the better seats we'll have. This certainly has a limit, but I don't think we're there yet, especially since we don't even have theater tickets.

The real value in Kayak wasn't that it included the big airlines, it was that it includes Southwest (not included anywhere else), Frontier, Airtran and many long tail airlines and forces the big airlines to compete. I traveled 250k miles in the past two years and often found myself on no-name airline flights that I would have never heard of if it wasn't for Kayak. And it has a super clean UI, isn't spammy and doesn't charge fees.

Our value is/will be: clean UI, no TicketStumbler fees (we're free!), no Ads on the main site, simple to use, natural language search and user dashboards & interactive maps (both coming soon).

<comment><USER6> _It's valuable because the more ticket providers we have, the cheaper prices will be and the better seats we'll have._

Yeah, that's true. What I'm saying is, that by itself isn't enough to be successful. Because of your next point, actually:

 _I traveled 250k miles in the past two years and often found myself on no- name airline flights that I would have never heard of if it wasn't for Kayak._

That example is a very different from a preppy highschooler buying a ticket to a concert. For one, you _had_ to travel. If all of a sudden there weren't any airplanes, it would still be necessary for you to travel. But it's not necessary for people to go to concerts.

Also, consider the number one reason why people go to concerts: because their friends are going. So one way to differentiate yourself from the competition might be to offer group discounts, for example. You could even sell them at a loss for awhile, just so you can use that as a way to market Ticketstumbler. Like "Going out tonight? Stumble into your next concert with Ticketstumbler. The more friends you bring, the more cash you'll save."

What I'm saying is, try not to hyperfocus on making Ticketstumbler like Kayak. It isn't, and can't be. So those types of comparisons are a good way to communicate to someone about what Tickestumbler does, but are a bad design way to design the company.

<comment><USER2> _> For one, you had to travel._

Not true. 100k of those miles were done for personal stuff (family, friends, vaca, etc).

The number one reason people go to concerts is because they like music, but I see what you're saying. The challenge isn't just helping people find friends to go with, it's helping them find music to listen to and making them aware when their favorite bands are in town.

The aforementioned user dashboards and interactive maps will eventually have everything else you mentioned.

We've never "hyperfocused" on making TicketStumbler like Kayak, which is why we have four blogs, a news site, directory listings (not just search), a video site and are much more open about what we do than Kayak. Also, one of our next projects will be completely out of left field (as to how we relate to Kayak). I think you really get what we're trying to do though so thanks a lot for your feedback. If you have anymore suggestions my email is in my profile.

<comment><USER6> Hehe, I'm sorry if I came off as critical. I really respect what you have achieved with Ticketstumbler.

Yeah, people go to concerts to listen to good music. But in the same way you don't typically go to a bar just to get drunk, people don't typically go to a concert just for the music. Socializing with friends is a very strong force that compels people to actually drop $50+ on a concert ticket.

Also, it probably won't be beneficial to try to help find new friends for people to go with. IMO, it would be kind of creepy to stand next to a total stranger for an hour if it "felt like" we should be talking and getting to know each other. :)

<comment><USER7> Actually, the only reason I ever go to a bar is to get drunk. Perfect record!

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> Can't comment on the photo, but I think "out of context" is a little harsh. Here's why: <http://tinyurl.com/5bvwzz> -Galen (the MHT reporter)

<comment><USER2> Yeah I just wanted to make sure everyone had the whole story. I certainly understand why it happens (I wasn't mad or anything) and how articles get trimmed down often outside the writer's control.

I just didn't want people to think we'd never ever take money. For angel funding you often don't need a model and sometimes don't need a deck. Rest of the article was great (even if the picture was a bit embarassing :-D). The other clarification points were for the Hacker News headline (I didn't want people to think revenue or profits were growing 50% per month), the picture (ha) and to expand on points made within the article.

Nothing tops this picture, however: <http://is.gd/7yx4> (This one was at Tom's expense instead of mine)</comment></comment>


Why We Have to Go Back to a 40-Hour Work Week to Keep Our Sanity - <AUTHOR> http://www.alternet.org/story/154518/why_we_have_to_go_back_to_a_40-hour_work_week_to_keep_our_sanity

<comment><USER2> Single page link for those who do not want to read an article split over seven pages:

[http://www.alternet.org/story/154518/why_we_have_to_go_back_...](http://www.alternet.org/story/154518/why_we_have_to_go_back_to_a_40-hour_work_week_to_keep_our_sanity?paging=off)</comment>


Ask HN: Is there a spectrum of difficulty/prestige in programming? - <AUTHOR>

I'm a CS student and I've been wondering about this. It seems most professions have this type of thing.
For example in engineering you would have chem eng at one end and maybe mechanical or something at the other. Doctors would have a scale that goes from, say, brain surgeons to (just guessing) podiatrists. In the military it would go from (probably) fighter pilots or Delta Force to basic infantry.
My question is, Does the programming/software eng. profession have a similar scale (or the perception of one, which is basically the same thing)? Or in other words, what would be the opposite of writing unit tests at your local insurance company? A web startup? AI? Trading bots? <comment><USER2> In terms of prestige, I think it goes something like this (from most to least prestiges)

0) Turing

1) Kunth/Pike/Ritchie/Englebert/Kay/Steele

2) People who wrote a language or operating system people actually use. Linus, Matz, Wall, van Rossum, Hejlsberg etc.

3) Theoretical computer scientists who have developed stuff people actually use

4) Low level specialists: security specialists, people who work on operating systems and/or performance specialists

5) People who developed a programming framework in wide use today, and have good taste (Cutting/Williamson/Rod Johnson/DHH)

6) Developers on apps used by huge numbers of people. Core Windows/Linux/Parts of Google/Parts of Facebook/Parts of Apple/some embedded systems

7) Theoretical computer scientists who've never developed anything used by anyone beside themselves

8) Developer at a software company or open source project with a good reputation for writing solid software (Google/some Apache projects/3rd party Webkit committer/some parts of Microsoft or Oracle)

9) Developer at a software company (ie, software is the company's primary focus)

10) Developer in some Financial Services companies

11) Enterprise developer/Web developer

12) VB6 developer/COBOL developer/Pick programmer

13) Developer in some proprietary language

14) Your work features on <http://thedailywtf.com/>

<comment><USER3> I'm not sure 4 is in the right order, considering you didn't name anyone/anything. I would put kernel dev's above framework devs. 0 and 1 are equal in my mind. Basically you should be able to give names for up to your #8 if you can't name person/company/project it should probably be lower.

I would order 0 1 (etc), 2, 3, 5, 8, 4, 7, 9, 10 11, 12, 13, 14 (comma placement is explicit where I find those levels equivalent)

<comment><USER4> Would Tim Berners-Lee or DJB fit as a good example of #4?

<comment><USER2> DJB would. I was thinking of <http://www.google.com/search?q=Dmitriy+Vjukov> when I thought of that category. Also Tomek ([http://www.topcoder.com/tc?module=SimpleStats&c=coder_ac...](http://www.topcoder.com/tc?module=SimpleStats&c=coder_achievements&d1=statistics&d2=coderAchievements&cr=144400), T.O.M.E.K from <http://www.mcplusplus.com/downloads/>)

Tim Berners-Lee is _maybe_ 3, or maybe category for people who "got lucky once and are now famous". Or maybe there should a separate category called "semantic web specialists". That may or may not be the same category as "Don Quixote, and other windmill tilters"

</comment></comment></comment></comment><comment><USER5> Your examples seem slightly wack:

"For example in engineering you would have chem eng at one end and maybe mechanical or something at the other"

OK, I don't know from engineering, but mech is pretty high up there. The low prestige engineers are probably civil engineers, not mechs.

"Doctors would have a scale that goes from, say, brain surgeons to (just guessing) podiatrists."

Fun fact: the low end of prestige is probably GP's. Podiatrists are specialists, and there's prestige in that. Some of them even perform surgery, which is worth considerable prestige. Very high prestige goes to medical researchers. ("Research" is a high prestige modifier to many fields. Consider engineers vs. research engineers.)

"In the military it would go from (probably) fighter pilots or Delta Force to basic infantry."

These actually depend on the individual military service, aside from certain generalities (combat people are higher prestige than support people, command is higher prestige than staff). As a result, infantry would actually be higher prestige than, for instance, mechanics or supply.

Army, I couldn't speak for. I know Rangers and Special Forces and Airborne are higher prestige than "leg infantry", and the cavalry were high prestige in their era, but I can't say for sure.

Navy is hard to say, since there's a rivalry between aviators, surface guys, and submarine guys. But all three of those can be unrestricted line officers, and that's the higher prestige track since it leads to command. If I had to hazard a guess I'd say the aviators have the edge--they command the biggest ships, and ever since WWII it's been demonstrated fact that the real power of the Navy lies in its air.

Air Force is run by the pilots, obviously. Combat pilots over cargo pilots.

Believe it or not the highest prestige component of the Marine Corps is probably the infantry. Everything else is there to support the infantry. Added to this is that every Marine is trained as a light infantryman, and every Marine officer is trained to command a rifle platoon. The Marine Corps doesn't really operate on these terms much, as they simply consider the entire Corps to be the best part of the military.

<comment><USER6> Podiatrists are doctors to the same degree that dentists are doctors. (Which is to say, they don't obtain the same degree obtained by physicians.) I think that research prestige is orthogonal to clinical prestige. Even then, I think that clinical fields do not have a linear order of prestige. It's more like individuals carry their own prestige (for research or for patient care).

</comment></comment><comment><USER7> Here's a hierarchy of programming languages (meant to be tongue-in-cheek): [http://blogs.msdn.com/blogfiles/steverowe/WindowsLiveWriter/...](http://blogs.msdn.com/blogfiles/steverowe/WindowsLiveWriter/ProgrammingLanguageHierarchy_1489F/programmer_hierarchy%5B7%5D.gif)

<comment><USER8> Haskell (omitted) probably comes in somewhere at the top :)

<comment><USER9> _gasp_ what about erlang?!

i like erlang :(

</comment></comment></comment><comment><USER10> I've found kernel and embedded programming to be the most difficult, particularly when there are big-endian/little-endian issues. Porting Linux to a new SBC is challenging, as is writing the board support package for it. Also fun is having to squeeze the last ounce of performance out of a CPU that some dingbat hardware engineer decided to use because it was cheap.

Prestige depends on how many people will die if you make a mistake. I've done medical equipment and air traffic control software. The sensible companies require at least twenty years experience before they'll even let you touch the code.

</comment><comment><USER11> I don't think Chem Es have any more prestige than any other major. They may net a higher salary, but that doesn't mean anything. My Chem E friend makes more than me, but he works in a plastics factory monitoring gauges all day with no design work involved. That doesn't seem very prestigious.

</comment><comment> daniel-cussen Maybe Wall Street quants and startup founders on one end, with Fortune 500 IT individual contributors (hardly what I'd call computer scientists) on the other.

<comment><USER12> Or kernel/compiler/AI hackers on one end and clueless Java school grads building "enterprise grade software" on the other.</comment></comment>


How Apple can win back the Chinese smartphone market - <AUTHOR> https://www.techinasia.com/apple-win-chinese-smartphone-market <comment><USER2> Personally, I think Apple will have huge success in terms of iPhone 8 sales over in China. Despite the fact that 8 is a lucky number, which is obviously a contributing factor, I also believe that Apple will win back the hearts of the Chinese market with the 8 after all the mishaps and screw-ups from other smartphone companies throughout 2016.

Whilst both Oppo and Samsung (The leading tech companies in China) received significant numbers of reports of 'exploding phones' \- Apple received significantly less of these reports.

I believe that now that the Chinese market has experienced what both Oppo and Samsung (who have both been industry leaders in the China market) have to offer, they will swiftly make their way back onto Apple's devices for the iPhone 8.</comment>


Ask HN: Review my Twitter App (ReTweetize.com) - <AUTHOR> http://www.retweetize.com/

<comment><USER2> It feels like a spamming service to me, just with a twist to make it less detectable.

<comment><AUTHOR> Thanks.

Its a reciprocation tool. But it does give people choices to retweet the messages they like. They don't have to retweet all the messages. They can read 20-30-40 (upto unlimited) messages and select just 4 of them to be retweeted per day. If they like none of the messages - they retweet none.

So how is that spamming?

<comment><USER3> Maybe you should keep track of messages that people choose to retweet and also assign karma to the people who originated the tweet so you can figure out which tweets are not liked by your users and maybe you can improve the 30 tweet candidates people get to choose from.

<comment><AUTHOR> Thanks for suggestions.

Right now, which tweets are not liked by users is being recorded. But the data is not being used to rank the tweets. In the future maybe.

Also, if the app succeeds, in the future - may add categories or tags or something too. So folks who like web development will only see web dev RTs. So folks will get to read more relevant RTs. But right now, adding all these features would be an overkill.

</comment></comment></comment></comment><comment><USER4> So you are encouraging people to spam their friends?

<comment><AUTHOR> People can choose the messages they want to RT. If they don't like a message, they can hide it.

Its not like mag-pie or other services where people have no control over which messages are retweeted.

<comment><USER4> I despise RTing as it is. Say something original rather than copy others.

As such I also despise any service that encourages people to RT, especially if it is disinteresting spam.

You've obviously put time and effort into this app. Wouldn't your creative energy be better spent building something useful, delightful, helpful, etc rather than a spamming tool?

<comment><AUTHOR> Fair enough.

You despise RT. But a lot of others don't. This tool is for them. Of course - you can't make a tool that makes everyone happy. And people who despise RT-ing is not the target audience.

But how is the tool spamming? People who like RT-ing can login. They can select exactly which messages they think are a good fit for their followers. And RT it. The messages they don't like, they don't RT.

Its a reciprocation tool - yes. But why would reciprocation = to spam if its not forced reciprocation?

Ulterior motive: And I am spending time building more apps that are useful, delightful, helpful. If this app succeeds, it will help me promote all those other apps too.

<comment><USER4> Your service is opt-in, people choose to RT your messages. That's seemingly fine.

The problem is twitter is a broadcast service. If one of the people I follow chooses to use your service then I also get to see your advert. Admittedly this is why it benefits you.

It is however highly irritating. I'd rather not have to unfollow an otherwise interesting friend because of a service like yours.

<comment><AUTHOR> I get the problem you have with RTs.

But thats why I've added tools that helps in getting rid of abusive / spammy messages from the network.

So hopefully people who you follow are kind and smart and will only RT interesting messages. A lot of choices are given to them to RT only messages that they think will be interesting for their followers.

</comment></comment></comment></comment></comment></comment><comment><AUTHOR> Not sure if its cool making an offer here... but if you run a startup, send me your twitter id after logging into ReTweetize.com (My twitter id = @ankeshk or email me ankeshk AT ankeshkothari.com) - and I'll add 20,000 points to your account. Not a lot but should get a few hundred visitors to your startup site.

</comment><comment><USER5> Unless you've got a way to monetise this, I'd give up right now.

<comment><AUTHOR> Thanks Jim.

Didn't build the tool for monetization.

Built the tool to make promotion of my other sites and apps easier. (Yes thats my sneaky ulterior motive :p)

If this app succeeds, getting initial seed traffic to other apps would become way easier too.</comment></comment>