Crawling the web at $2 per million pages - <AUTHOR> http://www.80legs.com/whatitis.html

<comment><USER2> I tried 80legs a couple of weeks ago when it was still in beta. It works well if you're willing to write Java code to process HTTP responses (see <http://80legs.pbworks.com/80Apps>). You can issue new requests in your processing code and write processed data to files which you can download once the crawl finishes. Even without Java, you can perform basic keyword/regex matching on crawled data literally by filling out a web form. And they have an API of course. Very impressive.

However, if you want to run crawling on your own infrastructure, I recommend Scrapy (<http://scrapy.org/>), a python crawling framework introduced on HN last year. Scrapy solves some of the more time-consuming problems involved in writing a crawler from scratch (multiple simultaneous requests, pipelined processing, raw caching, duplicate URL filtering) and comes with nifty development and administration tools. More importantly, it has an active and helpful set of core developers and good documentation. I am comfortable with both Python and Java but I chose Scrapy over 80legs because I can crawl for free on the machines I already have and I can afford to spend more time crawling from a single IP compared to 80legs which will let me crawl much faster but isn't free. Also, with Scrapy my bot can be 'naughty' - 80legs jobs obey robots.txt and limit the crawling rate per domain.

When my crawling needs outgrow my infrastructure, I am going to look at 80legs again.

<comment><USER3> Just a quick heads up - we'll be looking into allowing JRuby and Jython fairly soon for when you create 80Apps.

You'll also be able to mashup third-party apps and libraries into your own code. This is something we hope to have in 2-3 months.

<comment><USER4> How come there isn't a REST API? Java, JRuby, Jython. Ewwww. This isn't how outfits are doing things anymore...

<comment><USER3> The crawl runs on a super-heterogenous network of computers from around the world. The JVM sandbox is the only thing we know. Java is the only language that has something like this.

Yes, Java isn't as sexy anymore. But running your code on 50,000 computers sounds sexier to me than a REST API.

</comment></comment></comment></comment><comment><USER5> I would love it if it processed, stored, and indexed the data.

<http://parselets.com/> \+ mysql + sphinx

<comment><USER6> Wow - parselets.com is amazing. Thanks for sharing.

I think 80legs can do some level of parsing now, but Parsely seems like a great way to describe the analysis for targeted crawls on predictable data. If it supported the combination you've described it would really open up some cool possibilities.

</comment></comment><comment><USER7> That reminds me about my fools day joke about MassiveClouding. I imagined about a company buying cpu/hours from regular desktop computer users and selling it to anyone interested in this kind of cpu power. Of course any computer software could be run on the MassiveClouding without any changes. My good imagination :) I don't know about the technology behind 80Legs but these 50000 computers might be a bot net.

<comment><USER8> Isn't that like what the SETI project does? (Except that they don't buy the cpu power, they ask people to volunteer)

<http://en.wikipedia.org/wiki/SETI#SETI.40home>

"Any individual can become involved with SETI research by downloading the Berkeley Open Infrastructure for Network Computing (BOINC) software program, attaching to the SETI@home project, and allowing the program to run as a background process that uses idle computer power."

<comment><USER7> Yes it's something like it. There is another base difference; SETI@home or other folding projects have a centralized infrastructure for the data to be collected and bound. And all the software to be run on the client side are written for a specific mission. So every software should be rewritten for these platforms. My MassiveClouding dream has a some way of dissecting any software into parts and run on the clients. That's why it's a dream.

</comment></comment></comment><comment><USER9> You know, I'd try this, but the hard part is that the _processing_ is still on me (and I can't code, so I'm SOL). That, frankly, is the hard part, is it not?

<comment><USER10> Hi, I work for 80legs and wanted to address this--we will be building an "App Store" over the next couple of months that will allow you to use Apps written by other developers to do some pretty cool stuff. We already have several of these 80Apps (as we call them) available, many of which have been written by the semantic search engine company swingly.com. You can view descriptions for them on the "Create Job" page if you sign up to use our Portal.

</comment></comment><comment><USER11> I remember they had an explanation about this, but I don't remember what it was, and it seems to have disappeared: what are these 50,000 computers they're using?

<comment><USER12> Plura has a java applet which you can embed into a webpage that gets viewed for a long duration, such as a game or a streaming video. Affiliates embed an iframe which loads their applet as the user plays their game. The user's CPU goes up a bit, and they can help generate revenue for the game makers.

<comment><USER13> They're targeting desktop apps. The Java app downloads the pages, so it needs high permissions, so you get the default Java unfriendly popup asking you for confirmation.

<comment><USER12> From <http://pluraprocessing.com/developer/index.html>

      Plura supports desktop and web-based games.

      If the game is hosted on a website, like a Flash game, the
      developer only needs to include 1 line of iframe code. We
      will soon be releasing a Javascript API for dynamically
      controlling how this iframe is loaded, giving the developer
      control over starting, stopping, and controlling CPU usage
      in Plura. The iframe loads a Java web applet, which runs
      completely in memory. This applet is forcibly restricted
      from accessing the user's computer by the sandbox model
      provided by Sun.

See also - <http://pluraprocessing.com/developer/index.html>

So far as I understand, they have multiple models. One affiliate model is "Plura for Java Applets", where-as another is "Signed Java Applets"

I imagine there may be fewer options for unsigned applets, leaving the developer with less potential revenue every month, where as desktop application developers and signed java applications are left with the providers who don't need signed applications.

That said, I agree the Java dialog is ugly and scary ;( If I were an Affiliate, I'd want to avoid it. It breaks the user-experience of your site into some gaudy and jarring, not to mention unbranded and unrelated to the information the user is after.

</comment></comment></comment></comment><comment><USER14> I wonder how they arrived at their $2 per million pricing model. Why not $10 per million? Either one sounds ridiculously inexpensive.

<comment><USER15> They're probably using EC2s and then it's a function of how many pages can be scrapped per EC2 compute hour and then taking a premium on top of that.

The reason why they didn't want it to be too expensive (ie. 5 times as expensive) is (1) competitors can easily equilibrate and steal market share if their idea works b/c of the economic inefficiencies in their pricing model and (2) this game plan is more a game of dependence rather than up front profits, so it makes sense to take very little profit up front to get user traction.

<comment><USER3> We're not using EC2. AWS doesn't and can't scale to our size, actually :)

Side note: AWS guys have asked if we use them. We said "No, we'd be losing money then and wouldn't be able to scale to our size with you."

</comment></comment></comment><comment><USER16> One limitation is the throttling for each domain. If I had a smaller set of pages that I wanted to read frequently, this solution would not work. I understand the need to avoid DOS attacks, but in some cases, it would nice to be able to read a million pages from 3 large domains instead of 1 from a million pages.

<comment><USER17> That and...

"Your parseLinks() and processDocument() methods must complete within a total 10 seconds per document processed"

... as a limit on processing leaves room for competition. One advantage of the BYO-Cloud solution is that you can pay for more intensive processing of the crawl.

</comment></comment><comment><USER18> They're inflating the cost difference. Cloud : $0.10 / CPU-hour for "large scale" crawling. There's no reason to use a small, unreserved EC2 instance for large scale anything? I have a small reserved instance I use for git and bugzilla and so forth, it's slow as hell. That's why there are bigger ones.

<comment><USER3> The cost savings are still there with reserved instances if you do the math.

Actually, the cost is not the biggest issue with the cloud. If you're talking about large-scale crawling, AWS will not adequately scale. You can't get enough nodes or enough bandwidth.

<comment><USER18> My point was just in questioning a $4/million figure using a 0.10/hr instance. Why don't you run benchmarks against a reserved high CPU instance for 100M pages crawled? I was never contending that you cannot offer cost savings over custom crawling in ec2, just the way the #s on the page were calculated. If you have enough traffic to keep your instances saturated of course it is more economical to buy dedicated servers, but nobody starts out that way. Why in the world do you think you can't get enough nodes or bandwidth to do web crawling off of EC2? I remember seeing Bezos talk about animoto scaling up to 3500 instances instantly for video transcoding, I find it extremely hard to believe such a parallelizable task as web crawling could not be done on EC2. It's one thing to say that you can do it cheaper, it's another thing entirely to say that it can't be done on EC2 at all. How did you come to this conclusion?

<comment><USER3> The $4/million actually is not based on compute time, it's based on data transfer in/out of AWS. If I included compute time, it would be higher.

And it's the bandwidth aspect that makes web crawling not feasible on AWS. Yes, you have a few thousand nodes, but they're all going through a handful of external IPs, which will cause serious performance issues.. the worst case is that you'll get blocked entirely from the sites you're trying to crawl.

In other words, the bandwidth is not parallelizable on the cloud.

<comment><USER18> I'm really having trouble with this. My understanding is EC2 provides an internal and external IP for each instance : [http://docs.amazonwebservices.com/AmazonEC2/dg/2007-01-19/in...](http://docs.amazonwebservices.com/AmazonEC2/dg/2007-01-19/instance- addressing.html) as well as a semi-friendly DNS name. Each of these machines can certainly make its own requests to arbitrary URLs? I don't see how this is any different than a bunch of machines sitting in a data center with a shared, dedicated internet connection? Also, from my rather limited experience of crawling sites the only time I drew negative attention was when I did not throttle my crawler. If you are properly caching to avoid repetitious requests and throttling your requests, how are you going to get blocked and why would it be different in EC2 versus a dedicated hosting center?

<comment><USER3> So a few points (I assume you're talking about the Elastic IPs):

1\. Yes, each instance can have its own IP, but by default, each account is limited to 5 IP addresses.

2\. You can increase your limit, but my guess is that it's difficult to do so. You have to put forward a special request and have it approved.

You're right that blocking may not be a big issue, but crawling several different domains quickly will be hard.

Just so you know, we haven't encountered anyone doing large-scale crawling that considers AWS or the cloud in general to be a realistic option. The biggest reason is still the cost.. the outbound transfer rates just don't make sense at scale.

<comment><USER18> Elastic IPs are about having the same functionality as static IPs, every instance has an IP per the previous link I posted. Every time you connect a new network device to any typical network, it gets an IP. I'm not sure how that relates to scalability of the bandwidth.

You are limited to some number of instances (20, 50?) and yes, you have to fill out a form to get more. The previous example with animoto shows how far you can go. I would wager that finding the funding for a large # of instances is more problematic than getting the approval.

I don't see why crawling several different domains quickly will be hard? There shouldn't be any difference between a bunch of instances on EC2 and a bunch of machines in a data center, from a technical point of view.

As far as the cost argument goes, of course I agree with you. If you can project a high level of CPU/bandwidth usage for an extended period of time then of course you should buy dedicated servers.

The only argument I was trying to make was that it is completely possible to do crawling on EC2 or any other cloud provider from a technical point of view, the only limitation is cost. I see the advantage of utility computing is that it offers a cheap way to handle bursty traffic, which you may certainly run into if your server utilization projections are off? I don't think you should use it as your primary set of servers if you can project some large volume of traffic.

<comment><USER3> You're right that the data center and the cloud will be very similar, but our assertion at 80legs is that _both_ are very poor choices.

I'm not arguing that it's impossible to do crawling on the cloud. I'm saying it's near-impossible to do it on a large-scale on the cloud. 3500 instances is pretty good, but will still be an order of magnitude slower than what 80legs is capable of.

Now, if you show me someone that has 10,000+ instances on the cloud, I may agree with you!

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER19> This seems to be a very neat idea. How about processing of the crawled data, if someone wants to process the data and show it to the end user in a comprehensive way? We are looking for a solution which is affordable and provide both.

<comment><USER20> Stuff the results of the crawl into Hadoop, write some MapReduce jobs to process the data, and then serve the structured data back up with HBase?

</comment></comment><comment><USER21> ..:: Loop Detection?? ::..

Are you supposed to implement your own cycle detection?

If not, how deep is the cycle detection that 80legs offers?

\--

There are plenty of HoneyTrap[1] OSS projects which will quickly rack up lots of $$$ if the 80legs spider is backlisted.

For those who don't know, these projects create deeply linked pages, and sometimes create infinite cycles. They are trying to hinder spammers, but may hinder 80legs too.

\--

[1] [http://www.davidnaylor.co.uk/stopping-bad-robots-with- honeyt...](http://www.davidnaylor.co.uk/stopping-bad-robots-with- honeytraps.html)

<comment><USER3> Because of the way 80legs handles crawls, users don't need to worry about loop detection. There are really two issues here:

1\. For each user crawl, we only allow the same url once, so any simple loops that involve the same urls are eliminated by this process no matter how large the loop

2\. For more sophisticated "spider traps" that work with different urls and domains, they can have a limited effect on your crawls. Because of our per- domain rate throttling, the worst these traps can do is add a few cents per day to someone's crawl.

</comment></comment><comment><AUTHOR> MIT Technology Review also has an article on 80legs. Accessible at <http://www.technologyreview.com/web/23528/>

</comment><comment><USER22> Pricey!

I'd pimp for Open Google instead.</comment>


Ask HN: What html/css framework do you use? - <AUTHOR>

I've been dabbling in the Zurb foundation framework lately and it is great for all the projects that I have going on. Are there any that I should familiarize myself with? <comment><USER2> Foundation is my go-to. Good CSS frameworks are the 960px grid system [1] and the 1140px grid system [2].

[1] <http://960.gs/>

[2] <http://cssgrid.net/>

</comment><comment><USER3> Twitter Bootstrap is pretty much everyone's de facto CSS framework these days.

<http://twitter.github.com/bootstrap/>

</comment><comment><USER4> When doing rails work, HTML => Haml When doing perl, HTML => HTML::Template

For both CSS => Sass</comment>


Ask HN: Looking for books about raising kids - <AUTHOR> I'm looking for books which describes the intellectual and emotional development of kids from age of 3 to adulthood (and anything in between). Examples of successful and unsuccessful restraining /punishment/rewarding systems and the psychology behind it, the how's and whys. Maybe books or lecture notes for teacher or psychiatrist students.
Do you know any good books on this subject?
I searched for previous submissions but all I could find were "don't read parenting forums and give lots of love" but that's not what I'm looking for - toddler age is great but now I have to raise them into a functioning adult and I could use some handles. <comment><USER3> Parent of 2 young kids. Wife got this book from Amazon [0] called "How to Talk so Kids Will Listen...And Listen So Kids Will Talk" which surprisingly is really good.

[0] [https://www.amazon.com/Talk-Kids-Will-Listen- Listen/dp/07435...](https://www.amazon.com/Talk-Kids-Will-Listen- Listen/dp/0743525086)

As a parent, I have observed that kids observe YOU. Plain and simple. You want them to do something? You do it first and show them. For example, eating together as a family on Dining Table. My wife and I would just eat on the sofa and kids started expecting the same. They would run around during dinner time. One day, we decided to stick to a plan that we will eat only on the table. It took a bit of a time but kids are now eating on the table 90% of the time (not quite there with 100% yet but that is how it is with kids. No absolutes :))

So before you think about setting up systems for THEM, setup systems for YOU which they will just follow. Really hard as adults to change our own habits but it does wonders.

All the best. Raising kids is the hardest thing I have ever done and I have done a lot of hard things in my life. You will need a bit of luck, tons of support from family and most importantly: a clear head :)

<comment><USER4> I second this book. Whilst I'd heard some of the tips before, having it all in one place really helped me apply a considered approach to dealing with an oft angry toddler. Best of luck!

</comment></comment><comment><USER5> I don't believe you can learn this from a book. It's a highly personal journey, depending on you, your family and your kids. Your kids educate you as much as you educate them, you are in this together. I understand you want to seek advice but I honestly don't think there's a blueprint.

<comment><USER6> I agree.

One thing I would suggest is observe other people (and yourself), how they've turned out, and what conditions may have contributed to their current state (for good and for bad).

<comment><AUTHOR> Great suggestion but it won't work as it lacks data points.

Self reflection is my everyday bread since I have kids, but no one - <USER2> spouse - really tells you about their childhood traumas, his connections with his father or mother (and of course you will never hear the other end of a story) or how their parents were handling them, or have a chance to really see inside their lives. Without this information you cannot get to the reasons or good/bad parental decisions.

But a therapist could have this information. Could have enough data points. Could wirte a book about it. It would be a mistake not to read it and learn from others mistake and success.

<comment><USER6> What about those family friends who's kids turned out to be poorly adjusted adults? Or the brother in law who doesn't have clue about life. There are lots of places to obverse.

</comment></comment></comment></comment><comment><USER7> I tried to read a few articles/websites about parenting when my kids reached the three years. The problem is that most are about what at that time is the current fad in psych country. Most of that stuff is made up as they go without any scientific proofing what so ever. Never found a good one tbh.

I tend to follow the following guidelines:

1\. punishments generally don't work as a long term solution, only apply when temper is lost :-)

2\. Give lots of freedom so they can make errors. Only apply a rule to a situation when you can make a logical argument about it and make that rule situation specific instead of general.

3\. Spent time, talk to them, play games with them, make fun with them know what's going on in their live.

It's not perfect but so far they are decent individuals and i have lots of fun

</comment><comment><USER8> I am not sure if this is what you are looking for, but 'Mindset' by Carol Dweck is a good book. It talks about 'Growth Mindset' and how parents can inculcate this in their kids as opposed to the "fixed " mindset approach.

Another book that I would suggest is called 'Constructive living' by David Reynolds. Although that book is not about Raising kids per se, I am sure you can tweak some aspects of the book's teachings (for ex: Not allowing how we 'feel' determine our actions/lack of it) can be helpful when the toddler grows up.

</comment><comment><USER9> Pick a Montessori book for parents, the general method in a practical sense is get your child do things on their own and stay curious about learning.

How to Talk so Your Kids Will Listen is one book to consider for kids 8 or older.

I would highly recommend the How to Teach Your Baby to Read book and other books by that author. There are some nuggets of gold in there for teaching and this method works best starting at 12 months.

</comment><comment><USER10> Raising your children, raising yourself by Naomi Aldort.

An honest, down to earth and well tested (she has coached thousands of parents with "problem" children/situations) approach to natural, respectful and long- term parenting.

Natural - working with the nature of development and needs that a human child has. Working against them takes tons of energy and produces agony.

Long-term - playing the long game instead of looking for short term fixes resulting in "parenting debt" like addiction to rewards, broken trust etc.

Raising ourselves - because parenting is about growing up and often facing beliefs, trauma and triggers associated with one's own childhood. It is a great opportunity for self healing in the context of ending inter generational patterns.

Can't recommend it enough. Also recommend her talks — on YouTube and via her website.

</comment><comment><USER11> Oliver James: How To Develop Emotional Health [1]

Goodreads only shows a rating of 3.59/5 but personally I would recommend it. It speaks about how the early years are crucial for developing emotional health and what parents can do.

Disclaimer: I am not a parent so I can't tell how much it helps in practice.

[1]: [https://www.goodreads.com/book/show/20621101-how-to- develop-...](https://www.goodreads.com/book/show/20621101-how-to-develop- emotional-health)

</comment><comment><USER12> Unconditional Parenting by Alfie Kohn has a great perspective on the reward/punishment question. Doesn't check all your boxes but worth checking out alongside other options.

</comment><comment><USER13> "Parenting with reason" by Strahan, Dixon, and Banks. I liked that they actually cite evidence and it's not just someone's anecdotal ramblings.

</comment><comment><USER14> Farnam Street had an enjoyable interview with Barbara Coloroso recently.[0]

But, as others have said, there are no magic bullets, and much of the advice in this area is about fads. Like romantic love, every 'rule' has an exception, and it's easy to follow the rules over the edge to cruelty.

[0] [https://fs.blog/barbara-coloroso/](https://fs.blog/barbara-coloroso/)

</comment><comment><USER15> Check with your local Child Care Resource & Referral Agency, there's probably one in your area even if you aren't in the US. They will have great parent resources as well as very knowledgeable staff, some have their own Child Care Development libraries.

Here's a local CCR&R locator [http://childcareaware.org/ccrr-search- form/](http://childcareaware.org/ccrr-search-form/)</comment>


Ask HN: Successful examples of Productized Services? - <AUTHOR> <comment><USER2> I run [http://fairpixels.pro](http://fairpixels.pro) \- a productised service that helps engineers design better products. We've been profitable and growing rapidly from the get go. We're not trying to build a unicorn business. Just an environment where a small group of senior designers can work with a select amount of startups and do the best work of their lives.

If you have any questions about productised services, I'd be happy to share how we've done it.

<comment><USER3> Nice landing page! (Fellow web designer/dev here)

</comment></comment><comment><USER4> Here's a list of around a hundred of them: [http://jarhq.com/blog/100-productized-services- examples/](http://jarhq.com/blog/100-productized-services-examples/)

</comment><comment><USER5> We are a design studio called Superawesome, and we just launched Microawesome [https://microaweso.me/](https://microaweso.me/) a few months ago, which is a productized service kind of deal. We do tiny, fixed-price projects with quick turnaround. The kicker is we don't do revisions — we call it "what you get is what you get" — and this is where we're making savings for our clients.

We've learned over the years that we spend most of our time convincing clients to trust us and sign off on work we already know is good. Through Microawesome we're asking them to trust us, and in return they will get a good result fast.

It is definitely not for everyone, but that's why we're doing it "on the side", we still have our studio which offers a standard agency-type design service.

</comment><comment><USER6> I run [https://www.manypixels.co](https://www.manypixels.co) \-- It is an unlimited design service for startups. We also have a 10 days, 100% money back guarantee and can always add extra days if there are delays.

Here is a 25% forever discount code for HN (only 10 available): K4NT496

</comment><comment><USER7> I launching recently [http://www.bearglad.com/](http://www.bearglad.com/) This is a webdesign service. Currently i don't offer subscription service. I think there are many peoples who don't want pay monthly for only 1 design. Maybe in the future i offer monthly service for company if i have demand.

</comment><comment><USER8> This sounds pretty cool. I used to do recruiting, and designers were always hard to find, especially because most people didn't know how to hire a designer (including me).

</comment><comment><USER9> [https://resultspoint.com/](https://resultspoint.com/) \- B2B Lead Generation for IT and Saas Companies.</comment>


~ club - <AUTHOR> http://tilde.club/ <comment><USER2> The first rule of tilde club is ... We don't talk about tilde club</comment>


RESS, Server-Side Feature-Detection and the Evolution of Responsive Web Design - <AUTHOR> http://www.dmolsen.com/mobile-in-higher-ed/2012/02/21/ress-and-the-evolution-of-responsive-web-design/

<comment><USER2> What is "RESS?" The author doesn't define a title term, unless it's a corruption on REST via "Responsive Design + Server Side Components." Is someone trying to coin an acronym? I can't tell.

<comment><USER3> I probably should have done a better job of explaining RESS. Luke Wroblewski coined it as part of his initial article on Responsive Design + Server Side Components. I just assume the 'D' gets dropped somewhere along the way in creating the acronym ;) When I wrote the article I assumed most of my regular readers would be familiar with it. Luke's original RESS article: <http://www.lukew.com/ff/entry.asp?1392>

<comment><USER2> But still nothing to do with REST, right?

<comment><USER3> Nope, nothing to do with REST.

<comment><USER4> It's definitely the first impression one gets on hearing RESS and even more fun, it will basically be the same in verbal form.

So, maybe consider another term?</comment></comment></comment></comment></comment>


SMTP Strict Transport Security Coming Soon to Gmail, Other Webmail Providers - <AUTHOR> https://threatpost.com/smtp-strict-transport-security-coming-soon-to-gmail-other-webmail-providers/123789/ <comment><USER2> Does this mean self-signed CA's won't work with MITM?</comment>


Waze Mapping App provides roadmap of the future for app makers - <AUTHOR> http://interactionandflow.com/blog/2013/3/19/ive-seen-tomorrow-and-its-a-wazeian-world

<comment><USER2> waze is the most obnoxiously intrusive interfaces for open maps, but sadly the only one for mobile phones.

<comment><AUTHOR> I'd love to know what is obnoxious and intrusive? There certainly some room for improvement when it comes to the hierarchy of rerouting, vs user chat prompts vs head ups (police in 1 mile). But what do you have an issue with ?

<comment><USER2> you just described it :)

wife uses it. everytime i'm driving and she sets that up instead of navigator on my phone i go crazy how it likes to show up chat and whatever covering all of the screen with total disregard for the importance of instructions on the screen.

basically it goes like that: "turn .... in x miles" in my head "was that right or left, let me see the map" popup! traffic alert|police|accident|chat x miles down (but never mind you are going to leave this lane sooner)

or popup! something in 1 mile! click here if it's not there ... popup disappears BEFORE you are there.

or popup! something important in this route, click here to change routes ...popup disappears the exact second you are on a red light and finally able to click the change route button.

anyway. call me old fashioned, but that interface sucks for everything. i rater use a saner gps. and even if i have a passenger, i'd rather he/she goes writing down the gas prices in gasbuddy than fighting with waze's interface

<comment><AUTHOR> well if i worked with or for Waze, this feedback would go straight to their product / UX team</comment></comment></comment></comment>


Google Makes So Much Money, It Never Had to Worry About Financial Discipline - <AUTHOR> https://www.bloomberg.com/news/features/2016-12-08/google-makes-so-much-money-it-never-had-to-worry-about-financial-discipline <comment><USER3> So many companies try to emulate how Google works. There are multiple books, hundreds or thousands of articles. People speculate on the perks, the review structure, the hierarchy, the autonomy.

However, when other companies try to imitate Google they always fail because they're missing a crucial piece:

 _Billions of dollars in ad revenue_

Google doesn't work the way it does to be successful. It works that way _because_ it is successful.

I found this no more evident than when I worked on Google Search itself.

<comment><USER4> This is exactly correct. If you have a printing press that is dumping a couple of billon dollars into the basement every quarter in spite of what you're spending, you have the luxury of trying all sorts of novel things from a business perspective.

That said, the margins on the search advertising business are shrinking, and have been for years now. You can cover for that by eliminating profligate spending and then eventually you can't. I think Ruth is the right person to handle that transition but its going to hurt in terms of people feeling that "the magic is gone."

When that happens, Google is going to have to start rewarding operational efficiency gains at least as strongly as they currently reward popular science project type gains. And then it will be very interesting to see how they change and if the era of "but Google offers its employees ..." will be over.

I hope the free sodas stay though.

<comment><USER5> ""the magic is gone.""

Not only is the magic gone, but the boring, day to day fabric of functionality is gone.

Have you tried to _really_ search for anything with google in the past five years ?

I don't mean meandering, aimless, consumer-centric searches like "new fitness watch" where any result tangentally related is useful ... I mean a specific, meaningful, concise search with multiple keywords, all of which are _key words_ ... and the resulting pages do not contain one or more of those words.

I know all about allinsite: and quotation marks and so on ... those are, as far as I can tell, randomly interpreted by google. Who knows if they even work, ever.

A front page item on HN a few days ago asked what people would pay $1000 per month for ... I would pay that to have search with full control over search results with logical operators that actually worked.

<comment><USER6> I know exactly what you mean. "Half" "of" "my" "searches" "look" "like" "this" "these" "days". I can understand throwing out one of many keywords when I'm down on page 6, but when I search with only 4 terms and the very first result has 2 words s̶t̶r̶u̶c̶k̶ o̶u̶t̶ I start to wonder who exactly this search box is for.

<comment><USER7> True, and one other thing I've noticed is e.g. "Half of my searches" can bring say 200,000 results, but "Half of my" yields 50,000. A few other weird things that's been there for years and nobody seems to care. As long as consumer- level searches work and ads are clickable, Google's clock is ticking.

<comment><USER8> I wouldn't put too much stock on those numbers. They are probably statistical estimates, not actual counts.

</comment></comment></comment></comment></comment></comment><comment><USER9> Big companies seem to step in the same puddle, every time they try to set up one of these "breakthrough incubator" projects. They pick fascinating projects. They hire lots of smart people and give them great autonomy. But they inadvertently set up incentives that lead to the repeated creation of LINS (lavishly impractical non-solutions). And then the plug gets pulled.

I think the key problem is that the research team starts optimizing for periodic demo days with the boss. That's a sheltered environment in which clever (and easy to demonstrate) technical features are rewarded, and real- world annoyances involving customers, social norms, regulators, pricing, etc. are put off for "later." Not only are ecosystem problems not addressed, they aren't even vigorously considered.

The technology behind Google Glass was quite clever and some variant of that idea may eventually work. But what arrived on the market was LINS to an extreme. You can find similar failures in Detroit's concept cars, Paul Allen's first go at Vulcan, Xerox Parc's grossly overpriced STAR, etc.

I'm not sure how to fix this. Within the big-company budgeting system, it takes a daring CEO to allow super-innovative projects to break the usual rules about desired five-year ROIs. Once such a CEO takes a stand, it's really hard for him/her to get out of the way.

Suppose these CEOs do miraculously set up a system that approximates the scrappy, minimum-viable experiments of a true startup, with the marketplace being the true boss. That still is problematic. When a big company is pushing MVPs into the marketplace, the public scrutiny and scorn makes it really hard to recover from an awkward start -- and keep iterating in peace.

<comment><USER10> This is exactly what Alphabetization is meant to solve. A federation of companies can more effectively allow individual divisions to operate like startups. Additionally, it's a lot easier to push experiments when they're not attached to the Google brand.

Personally, I think Google should just funnel all the profits into GV.

<comment><USER9> Ah, but my point was that they don't operate like startups. They may adopt a lot of startups' superficial trappings, but in their first three years, they aren't sharpening their objective by being cash-constrained and exposed to the evolving tastes and distastes of the outside ecosystem. They're semi- generously funded while building whatever will amuse Sergey and Larry. That's creates LINS problems, again and again.

<comment><USER10> I agree that they don't operate like startups enough, but it is in fact the goal of Alphabetization.

You're right that it's not working well enough. That's why I think they'd be better off simply making GV the largest private equity/venture capital firm in the world.

</comment></comment></comment></comment><comment><USER11> There's an interesting fiscal corollary of having unbridled ambition matched by enormous firepower which brings the world's smartest engineers and hackers together: a Xoogler who was early in the business once told me about how engineers would hack/exploit the travel allowance policy.

Google allowed folks to book their own flight and accommodation, and had an algorithm which determined rewards for folks travelling with financial efficiency.

The measure of 'financial efficiency' was an invisible coefficient tied to average hotel and flight prices in an area. So if the average cost of a trip from Mountain View to Boston for three days was $1,500, and you managed to do it for less, then you got back a cut of the difference either as a cash bonus or in points to spend on upgrades for hotels and flights.

After a while someone realised that a group of engineers were consistently booking shitty travel around the same time, and then travelling first class the rest of the year.

They had worked out what the algorithm was doing, and started scraping hotel prices themselves. They booked (spurious) cheap earlybird flights and hotels during conference season in various cities, sometimes years in advance, and took huge numbers of internal points from doing so.

The person describing the situation to me said there was a lot of discussion as to whether they should be fired or rewarded.

<comment><USER12>
      > After a while someone realised that a group of engineers
      were consistently booking shitty travel around the same time,
      and then travelling first class the rest of the year.

How's that different than having an annual travel budget? Sounds like they sacrificed to feast later instead of holding accommodations constant at "above average."

I think you did a good job telling the story, but maybe I'm missing how gaming travel expenses is unique to Google.

<comment><USER11> Crucial point which I didn't communicate properly: they were block booking spurious/unnecessary travel well in advance (by years).

<comment><USER13> How can you even book flights years in advance when airlines only publish schedules 330 days in advance?

<comment><USER11> You can book hotels years in advance and flights ~1 year in advance.

</comment></comment></comment></comment></comment><comment><USER14> Contrast Google's old way of managing bets with Tesla.

When Google started working on self-driving cars, they went about it academically. They did not plan a sequence of stepping stones that they could sell. They meant to have a product in an indeterminate future which should be immediately perfect and better than a human, essentially not needing a wheel.

Tesla went about it with an engineering perspective. First, cars that can send accumulated data over the Internet. Then, equipped with cameras. Then, with limited assistance, warning the user to take the wheel back in difficult situations. And sporadic updates adding support for more complex cases.

The end goal is identical; but they make money along the way.

<comment><USER15> > but they make money along the way

Only killing a few people in the process, but hey who am I to judge?

<comment><USER16> Are you implying that Tesla killed the (1 person, not people) who decided to let his car slam into a tractor-trailer on cruise control?

It's not a problem to you that, say, Apple has "financed the suicides of dozens of Foxconn workers," or that GM killed 140 people by suppressing information on a defective ignition switch, or that Lockheed Martin and Halliburton buoy their stock by selling weapons to kill hundreds of thousands of people?

Evidently, you must find totally acceptable the gun-running, terrorist- sponsoring activities of HSBC in the last few years. I bet Google's permabanning of people from their own Google Apps accounts (because they were reselling a couple Pixel phones) is ethically fine with you, as well.

Never mind the tens of thousands of preventable automobile deaths yearly because car manufacturers refuse to implement even the most rudimentary of Autopilot functions.

Yeah, totally reasonable to pick on the electric car / solar battery company with $3b in revenue since one dope decided to drive into a truck and behead himself. There's _nobody else_ deserving of criticism whatsoever.

Let's not even get started with how cocky their CEO has been with his fancy sustainable business plans!

p.s. I won't even attempt an ad hominem at the guy selling homeopathics and 'nutritional supplements' in this thread.

<comment><USER17> I'm going to ignore the whataboutism you spew.

Also, Elon musk has done amazing work with electric cars, sustainable energy and space travel. No doubt about that.

But this is about Autopilot, a collection of driver assistance program. Not sure about OP, but my spite with Tesla is that they are pushing the above- mentioned driver assistance programs as self-driving features. This over- selling is going to kill people.

And for your information, lane keeping assist, lane driving assist, automated cruise control (which is pretty much AP1.0) have been present in several cars. Even self parking has been demoed by several cars. Add in automatic lane changing (which doesn't sound difficult to me), it wraps up Enhanced AP1.0.

Does that sound like a completely autonomous self driving car? Because that's what the public perception is like.

<comment><USER18> Please don't call names ('you spew') in arguments. If someone is wrong, explain so civilly.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

[https://news.ycombinator.com/newswelcome.html](https://news.ycombinator.com/newswelcome.html)

<comment><USER15> How is _" you spew"_ namecalling?

 _I_ was the one personally attacked in this thread, not the other way around. Just because my comment doesn't fit the Musk Worship narrative doesn't make me wrong.

Was someone _not_ killed by a malfunctioning / underprepared Tesla? What am I missing here????

<comment><USER18> "You spew" is a pejorative. Examples:

[https://twitter.com/search?q="you%20spew"](https://twitter.com/search?q="you%20spew")

[https://www.google.com/webhp#q=site%3Atwitter.com%20%22you%2...](https://www.google.com/webhp#q=site%3Atwitter.com%20%22you%20spew%22)

What you're missing is that even though someone was uncivil to you on HN, you needed to remain civil in return. That's the social contract here, and we need it to prevent discourse from degenerating even further.

<comment><USER15> Given that there is no report button or ability to downvote someone that replies to you (at least not on mobile for me), it is only reasonable to defend myself in that situation.

The ensuing attacker went to my profile, researched me and my business, and personally attacked me because they disagreed with my sentiment.

What is my recourse for action in such a case? Take it on the chin like a wimp? I don't think so.

If you don't provide appropriate tools to stop a tit attack, then you are fostering an unfair and "unsafe" environment by banning tit-for-tat response.

You are turning this community into a one-way echo chamber.

<comment><USER18> Your recourse is to email firstname.lastname@example.com, as the site guidelines ask. Note that I chided that person for what they did, which of course was wrong. Two wrongs don't make a right, though, and you're continuing to break the rules while they at least stopped.

For civil discussion we all need to hold ourselves to a higher standard than the one we perceive other people to be keeping, because the only alternative is a downward spiral, and an accelerating one. If you want to keep commenting here, we need you to internalize that.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER19> Xoogler here.

About a week after joining, and having partaken of the food, massages, meditation rooms, fitness centers, shuttle bus, sports facility, arcade, juice bar, tech stops, shwag etc. - I remember thinking to myself "whatever we do here must be ridiculously lucrative".

There weren't signs of obvious waste (at least in my org.), and coming from other corporate gigs where you had to pay for coffee it really opened my eyes to what a difference it makes for employee morale / productivity.

<comment><USER20> A lot of that stuff although is cheaper for the corporation to provide than for the employee to pay for themselves in money and time. The corp has a tax advantage and a bulk purchase advantage. Not having a payment infra probably adds a little bit of efficiency too.

<comment><USER21> Which does seem like sort of a loophole. Don't companies have to add perks over a certain amount to the employees pay so that it gets taxed? Wouldn't feeding an employee every day qualify?

It would be sort of weird if we turn back into having sort of 'company towns' where your employer provides a lot of what you need except they pay for it. It's not just Google this would be beneficial for, even if you run a factory you could pay employees a tiny bit less, but provide food/services for them that would make up for it. And because of the taxes/bulk situation it could be economically beneficial for both sides.

<comment><USER22> I one spoke with a woman from the Netherlands who was surprised that these perks are not taxable. According to her, in the Netherlands perks like shuttles to work or company-provide food are taxable.

<comment><USER23> What if the company had a business meeting and lunch was provided?

<comment><USER24> If it involves clients, then you need the paperwork and may save on part of the VAT + part of the money could be considered expense. Usually it can be quite complex and it's not harmonized in the EU.

</comment></comment></comment></comment></comment></comment><comment><USER25> _"No one wants to face the reality that this is an advertising company with a bunch of hobbies."_

Remember Nokia. They produced mobile phones with excellent quality, extreme ruggedness, good battery life, and good voice quality. Their manufacturing was highly automated and their costs were low. They focused on their core business. Recently, their CEO said "We did everything right, and we lost anyway".

On the other hand, Google has repeatedly failed to develop a second big money- making product. They failed at social. They failed at fiber-to-the-home. Android is a loss leader to drive ad traffic. Automatic driving may pay off, but that is being a component supplier to an car company, not a high-profit business. Google's second venture into smartphones might be a success, but that remains to be seen.

<comment><USER26> > Remember Nokia. They produced mobile phones with excellent quality, extreme > ruggedness, good battery life, and good voice quality. Their manufacturing > was highly automated and their costs were low. They focused on their core > business. Recently, their CEO said "We did everything right, and we lost > anyway".

I don't agree that not developing a side-business is what cost Nokia their throne.

Very simply put, it was because they rested on their laurels. A Nokia executive told me the original iPhone is going to fail and used the Nokia N95 spec sheet as the only argument. It was the most impressive smartphone of the time but he completely failed to look ahead, as Nokia had a huge market share, large R&D budget, their own factories and no real competition.

Of course the new platforms crushed Nokia while they had several half-hearted attempts to follow them. They were completely setup to be crushed by any disrupter with money and a vision.

<comment><USER27> One more reason because of which Nokia failed, I think, is because they had a platform which was somewhat successful. Symbian. They stuck with it too long rather than jumping the Android ship. If they had embraced Android whole heartedly, I think they could have competed with Samsung for the Android supremacy. Their hardware was good and they had loyal customers, at least in India.

</comment></comment></comment><comment><USER28> This is the killer quote:

"No one wants to face the reality that this is an advertising company with a bunch of hobbies."

It's so brutal but (IMHO) sums up Alphabet in one sentence.

<comment><USER29> However, many of these "hobbies" actively support ad traffic. Chrome and Android ensured Google's high market share for search & maps while delivering additional ad revenue channels.

By the number of projects most are side projects that make a loss, but the really big ones have a big contribution to Google's success (via ads).

</comment></comment><comment><USER30> Google has a fundamental problem with solving challenging issues, especially those in the realm of physical "meat-space". Really hard problems, problems that might reveal that Google is not omnipotent, where Google might not succeed, Google pulls back from. In areas where Google is, Google dominates the landscape. In all others, Google does not deign to participate. If Google can't _dominate_ a market, it is not interested.

> _Former employees say Page became frustrated with Fiber's lack of progress. > "Larry just thought it wasn't game-changing enough," says a former Page > adviser. "There's no flying-saucer shit in laying fiber."_

The issue with the physical world is that fundamentally, compared to a lot of software or hardware technology, it __is__ fairly boring. Boring work is also some of the most important work; the synchronization of traffic lights, the scheduling of flight paths, the minutiae of power line access, these are all boring, _important_ things. A common reaction to boring problems by technologists is to throw computers at them. Build an algorithm to synchronize traffic lights. Build an AI to schedule flight paths more efficiently. I understand these reactions, the impulse to replace existing, less-optimal processes with stronger, more efficient, automated ones. Google does a really great job at automating things, its their core competency. However, automation can't solve all problems. You can't automate away political processes. You can't automate away property rights and fair access to power lines, and contracting construction work. Page clearly feels uncomfortable with this, and so he abandoned Fiber.

Honestly I am not sure Page was wrong to abandon Fiber. Google's core competency is automation and if they can't automate a problem, maybe they _shouldn 't_ be involved in the space. Or maybe Google should get a new core competency and learn new things. Teaching an old company new tricks is hard though. Very hard. So hard that most companies fail before they can ever adapt. Without being able to solve problems in the physical space, Google is fundamentally "locked-in" to only being able to solve digital problems in the digital world. This is why Google has no physical stores, no physical products (which are not manufactured by someone else - Chromebooks, Pixel, etc do not count. Prototyping is easy. Selling to customers in a retail store is hard). Microsoft had the same problem as Google once. It was hard for Microsoft to learn how to run stores, build its own products; it took Microsoft over a decade. Let's hope Google can learn too.

<comment><USER31> > Really hard problems, problems that might reveal that Google is not > omnipotent, where Google might not succeed, Google pulls back from.

I've heard this phrased as "Google has the good fortune to be operating in a field where being 90% right 90% of the time is more than good enough"

If the search result you want is number 3 instead of number 1, that's totally acceptable. If your ad-targeting is good but not perfect, you can still make enormous amounts of money.

There are other domains where 90%/90% is absolutely unacceptable: avionics, medical life support, rocket propulsion, etc.

<comment><USER32> > "Google has the good fortune to be operating in a field where being 90% > right 90% of the time is more than good enough"

I don't think it's an accurate view.

It was good enough when they started. Now they are doing a lot better than that. The results are moved dynamically per user over hundreds of criteria.

It really is not a dumb search/ad engine that anyone could replicate easy. They have evolved to be damn efficient at what they do.

<comment><USER31> > It really is not a dumb search/ad engine that anyone could replicate easy.

I'm not saying it's simple or could be replicated easily. I'm saying that the marginal improvements they've made don't really matter that much to users.

I mostly use DuckDuckGo these days, and its native search results are markedly inferior to Google's, but it basically doesn't matter because I still almost always get what I want on the first page of results.

<comment><USER33> But it does matter to a lot of people, which I think is the thing.

There will absolutely be people who are willing to sacrifice some ease-of-use in the name of additional privacy, but it seems that those people are in the minority, and that means that Google (or whomever) needs to provide 99/99 when everyone else is only 90/90, if they want to continue being market leaders.

<comment><USER31> "But it does matter to a lot of people"

Does it?

Imagine some alternate reality where Bing were the dominant search engine (or look at real world cases like Russia, where Yandex dominates), do you think that Google's marginally superior results would be enough to make users switch?

I propose that Google reached a market-leading position early on by being pretty good when other search engines were pretty bad. Since then, they've become the "default" option, and people will keep using what they're accustomed to.

</comment></comment></comment></comment></comment></comment><comment><USER34> Yup. There was SOOOOOOOOO much bullshit work being done there that could ONLY be done because Google can afford so much top-notch talent to work on research, even if it reinvents the wheel. It was a big part of the reason why I left.

That being said, at least they didn't invest their dollars on loads of red tape, though to be fair it isn't like all of Google is incredibly regulated unlike many other multi-billion dollar companies

<comment><USER35> If you actually have a worthwhile idea or something you're working on starts going somewhere, the masses will descend trying to stall it out. They'll also magically try to slap their face/name on it, while trying to push you out.

Also, it's funny how many random things are being worked on, as it seemed impossible to get time or support to work on anything. I wonder what process/reasoning goes into deciding who gets to work on their own thing.

This company is so typical regarding many things that most of their PR/marketing sound like outright lies.

</comment></comment><comment><USER36> The robotics effort sounds like an absolute disaster. How shareholders haven't been demanding answers there is beyond me. Acquiring 11 companies to, apparently, satisfy Rubin's obsession with robots (it's named Android for a reason) and then trying to shut them all down or flog them off ... but nobody wants them? Yikes. How did they get so starry-eyed about Rubin that keeping him (also failed) was apparently worth such a huge effort? Yes, he ran a successful operating systems project and fully demands enormous praise for Android's wild success. But where are the limits to retention efforts?

<comment> compiler-guy Shareholders can demand answers, but it won't get them anywhere. For better or for worse, Larry, Sergey and Eric own Google and can do whatever they want.

<comment><USER29> Shareholders can sell and thus lower the stock price (and the owners' wealth). The fact they don't do it means that many people seem to believe in management's ability to earn future profits.

</comment></comment></comment><comment><USER37> Really? "Never"

What about 18 months ago?

> Google Earnings: Profits Soar As The Company Reins In Cost

[http://www.forbes.com/sites/greatspeculations/2015/07/20/goo...](http://www.forbes.com/sites/greatspeculations/2015/07/20/google- earnings-profits-soar-as-the-company-reins-in-cost)

> Google shares surge on talk of cost discipline

[http://www.usatoday.com/story/tech/2015/07/16/google- earning...](http://www.usatoday.com/story/tech/2015/07/16/google-earnings- second-quarter/30246847/)

Whatever, keep the narrative.

</comment><comment><USER38> I'm no businessman, but does anyone else feel this will kill a lot of potential innovation? Not every project pays dividends immediately, and some are worth pursuing even at a long-term loss. (imagine if we all had fiber today and bandwidth was ridiculously abundant. Just how different would the internet be, just as when electricity became abundant?) Also I understand that Google is a business, but the need for a business model for research might stymy forays into interesting tangents. (I can't currently list any, but history is full of examples of things that were invented while looking for a solution to a different problem) Sure a company shouldn't be hemorrhaging money or throwing it at anything that moves, but it also shouldn't be completely beholden to (my generalization of) investors who can be quite short-term-thinking and risk-averse.

<comment><USER39> Very much so.

Michael Dell effectively rescued his company from board room rampant shorttermism some years back because of just such a problem.

And we basically witnessed Nokia and HP effectively crash and burn because of boardroom demands for short term profits while the execs were trying to pull of long term "pivots".

</comment></comment><comment><USER27> I think this article is trying to unnecessarily create issue out of something very obvious. It's very clear that Google's main source of revenue is ads. But that is backed by very strong products which are an integral part of billions of people's daily lives. Search, You Tube, Android, Gmail, Maps and a few others. As long as they maintain their lead in these products, their ad revenue is not going to go anywhere.

What's even more encouraging is that most of these products are improving with great speed every year. They are fighting the Cloud Battle very hard. So I think it is OK that some of the big dreams they have tried like Google glass and autonomous cars have failed. One needs to take big risks to get big wins. I think if they stopped these moon shot projects then it will be a thing of big concern, then they would have stopped innovating. Innovation is what give the Google "brand" it's aura, which makes people believe that whatever Google produces must be good. It gives them the edge.

</comment><comment><USER40> I was surprised to see no mention of Bell Labs or Xerox PARC in the article.

</comment><comment><USER41> This thread being here today is rather ironically timed, given that Google announced internally today it isn't giving its employees any Christmas gift for the first time in its corporate history, presumably because of the financial impact.

</comment><comment><USER42> To some degree, this is like anything else in life, isn't it? The more successful/skilled/good you are at anything, the less you have to worry about whether you're doing things the 'right' way.

See for example, sports. A gifted athlete could quite easily defeat a far less gifted one with more 'technical knowhow' because they're simply more skilled by default and can hence make up for their inefficiencies or mistakes.

Or maybe any RPG game ever made. Come across a super powerful character or grind your party 100 levels higher than everything else, and it doesn't matter what tactics you use; you're simply too powerful to lose.

[http://tvtropes.org/pmwiki/pmwiki.php/Main/ScissorsCutsRock](http://tvtropes.org/pmwiki/pmwiki.php/Main/ScissorsCutsRock)

And the same obviously with companies, individuals and money. Enough money means even a lot of stupid decisions will work out well, whereas a competitor with much less will simply falter in the same situation.

<comment><USER43> >See for example, sports. A gifted athlete could quite easily defeat a far less gifted one with more 'technical knowhow' because they're simply more skilled by default and can hence make up for their inefficiencies or mistakes.

This is a surprisingly good analogy. Look at Steph Curry in the NBA. He takes a ton of awful long range 3-point shots, but he also makes a ton of them (45.4% in 2015-2016 compared to league average 35.4%). But some people are arguing that he's actually a detriment to the future of basketball because kids are emulating him instead of learning fundamentals:

>"Even watching this weekend, everyone is trying to hit the three and there are a number of big guys that are pretty good that aren't seeing the ball at all," Johnson said at an EYBL stop in Indianapolis in April. "It's like guards feel like they have to shoot the three to reach their goals, but it comes at the sacrifice of actually playing a team game. There's a lot of guys chucking a lot of bad shots without actually knowing how to get their teammates involved and create for other people."

[http://www.sbnation.com/nba/2016/6/13/11876304/stephen- curry...](http://www.sbnation.com/nba/2016/6/13/11876304/stephen-curry-youth- basketball-aa-mark-jackson-warriors)

</comment></comment><comment><USER44> One of the problems with companies that have cash cow revenue streams is that it sometimes doesn't matter what kinds of decisions the executives make, poor or good, the money will keep rolling in...until one day it doesn't and nobody has a clue how to actually manage the rapidly sinking ship.

I've been studying Atari for quite a while, there's so many interesting pieces to the company and their success and failure, and it's all been so well documented and autopsied that it makes for a marvelous case study that still has amazing applicability today.

<comment><USER45> I think you can look back and see some things that Google has done successfully. Youtube turns out to have been a good buy. Google successfully managed the transition to mobile, which many questioned whether Google would be able to do ("Mobile first"). Most recently, Google has made a huge bet that their future is intrinsically tied in with machine intelligence ("AI first"). We'll get to watch over the next 5-10 years how that plays out.

(It's probably already clear that there's a lot of power in an AI-first approach, but it's worth noting that this change didn't just happen with Sundar's blog post last month. A better way to look at it is to realize that if Google has been running their custom machine learning ASICs for over a year now, they must have started designing them well before that, which, to me, was amazingly early. Google acquired DeepMind in 2014.)

Not all those bets will, or have to, work, as long as some of them work out really well. I'm curious to see what happens with VR, for example. But I think AI is the most obvious central bet, in the same that that mobile was beforehand.

<comment><USER46> I'd bet that "AI first" will play out well for them, because AI is beneficial to both search and ads - while self-driving cars, fiber and balloons are not.

</comment></comment></comment><comment><USER47> Plenty of companies make so much money. Google didn't have to worry about financial discipline, not because it had so much money, but because it has an unassailable competitive position that is not aligned with its cost structure. Put simply, if you had the mandate to bankrupt a company through competition and had an infinite amount of money, I believe it would cost more to bankrupt Google than any other business.

</comment><comment><USER39> Best i recall, the Glass problem was more Brin over-hyping than the tech itself.

And i do not recall reading about the car getting into trouble while under computer control. Either someone ran into it or it ran into something while human operated.

<comment><USER48> The car actually did hit a bus while in autonomous mode. To me, it was an odd situation though. The lane the car was in was for parked cars as well as traffic and it was ending at a turn lane or something. [http://www.latimes.com/local/lanow/la-me-ln-video-google- car...](http://www.latimes.com/local/lanow/la-me-ln-video-google- car-20160309-story.html)

</comment></comment><comment><USER49> As the saying goes, Revenue Solves Everything.

</comment><comment><USER50> Success is a very cruel mistress.

Without some sort of feed back cycle - usually given by the market, things grow in strange and perverse ways fast.

The trick is to develop a feedback cycle that isn't as harsh as the open market (manage by quarters!) without giving absolute free hand.

</comment><comment><USER51> I think Google has been trying to somewhat imitate how life under nature selection works. Bring basic-but-extensiable-happy-route-scoped projects to life to test markets and wait and see which one would have the DNA for the next boom. From this perspective, I think they actually allocate their resources very carefully and intelligently. But the next emerging DNA perhaps needs more than just luck. Maybe more persistent and focused effort. I guess W. Brian Arthur's work has not small influence on its strategy. In fact, if I recall correctly, he is/was some kind of advisor to Google. Just my personal speculation.

</comment><comment><USER52> Google has some killer products that aren't just about adsense for the web:

Youtube - incredible future for revenue Google Docs/Spreadsheet/Slides - <USER2> future of productivity suites - easily monetisable when needed Android - dominating this space bodes very well

They are gently providing an entire computing stack that covers all a simple user might need: android/chrome OS/chrome browser/cloud

Yes they have a bunch of crazy hobbies. But they are dominating in a number of key spaces.

</comment><comment><USER35> The problem with it's lack of financial discipline is that the money isn't going to anything worthwhile. Everything they've advertised they're investing in that seems to be worth something either fails to go anywhere, or was just outright BS.

They essentially have a bunch of people doing nothing much in particular, going nowhere in particular, while acting on their whim and whatever their hand position happens to be on that day. It's even worse as they constantly try to claim they are about-- something greater.

It's hilarious. It's not that they are wasting money (ie, doing something worth something but that may not provide any returns, or something big and unconventional but that doesn't seem relevant at the time), it's that they are wasting money (ie, wasting money).

They are either doing what they're doing to make more money or to protect their revenue stream, or to be once again doing the dumbest thing.

Nothing really gets funding unless it's going to make more money, or serve as good PR to distract from how full of it they are or to try to brush aside failure.

These days, when their products fail, all I see is failure. That is, it probably took 2 years to develop whatever crap was released, rather than it being "early days" and them "trying many different things." They picked one thing, spent an eternity on it, pushed out absolute garbage, then failed.

Everything else that went somewhere or that could go somewhere gets caught in political gridlock, as the idiots descend to try to claim credit, while really trying to be the cliche they've always been: stomp out anything that's going anywhere, because... <insert whiny bitching here>.

</comment><comment><USER53> Google is an oil state. They setup shop and pretty much hit a cash gusher.

</comment><comment><USER54> Innovation usually happens when there is scarcity of resources.

<comment><USER55> Ain't that the truth.

</comment></comment><comment><USER56> I always wondered if Google just fired all their engineers except a few top guys, and focused on their core money making products like search / ads / etc. and tried to stay as lean as possible (lean being a relative term), how much profit they could make.

<comment><USER57> I concur. But then I tried to classify Google projects in 3 categories and find for "obviously useless things that only exist so engineers can have fun", and I didn't find so many obvious things:

\- Obviously productive ones: Gmail, Google Analytics, Android, Android TV, Maps, Youtube, OpenSocial, Chrome, because they all participate to "universally trackable users",

\- Skeptical projects: Fiber, Flights, Google Scholar, TensorFlow, Chrome OS, GDrive, GApps for Business,

\- Pet projects that may be fun but don't seem to serve as leverage: GCE (yes, their cloud solution), Google+, Go, Google Earth, Google Play/Music, Google Wallet, Google Glass, GGroups, GFinance, GTranslate, Google Fonts, Google Classroom, Inbox, Wave, Hangouts, Google News, Google Currents, Google Keep, Google Now, Wave...

Therefore... If they wanted to stay as lean as possible, would they have come up with the right list straight away? What makes products "obviously" part of the first list?

[1] [https://en.wikipedia.org/wiki/List_of_Google_products](https://en.wikipedia.org/wiki/List_of_Google_products)

<comment><USER29> Even the skeptical ones are probably productive:

\- Flights will likely make money, it operates on a comission basis. It's just not big compared to other projects

\- TensorFlow is part of the machine learning programme that is used heavily to make more money in other areas. Sharing knowledge helps to get talent in that area

\- Chrome OS is like Android for laptops. It ensures that you use Chrome and Google. The adaption rate is not huge, but I could imagine that it still pays off

</comment></comment></comment><comment><USER58> Brilliant article. Larry Page isn't impressed with Fiber because "it wasn't game-changing enough," and that "There's no flying-saucer shit in laying fiber." Definitely gives you a peak into the thought process of this giant.

</comment><comment><USER59> Funny, Microsoft used to talk this way in the 90s when IE and Windows ruled the roost. While they are still strong, they do now appear to worry more about financial discipline. Worrisome predictor for today's darlings?

</comment><comment><USER60> And it's not only ad revenue, it's also tax-avoided profits.

</comment><comment><USER61> That Porat person is a serious contender for buzzkill of the year. Google makes tons of money with ads and spends it on geeky experiments to try to push a futuristic envelope (and create an environments where they can sell more ads, like a self-driving car) we'll all benefit from eventually and she gets to be the person that locks the toys away because "financial discipline".

<comment><USER62> Buzzkill? Sure.

But she also sounds like one of the few responsible adults in the place with a hand on the purse-strings, whose job is to ensure that there _is_ an Alphabet and Google in 10 years, shiny toys be damned.

<comment><USER63> There will be an Alphabet or a Google in 10 years but it will look eerily similar to the aging behemoths of Silicon Valley today that take paltry swipes at different markets while their main revenue source slowly dwindles (up until the point they are bought by a holding company).

</comment></comment></comment><comment><USER64> > The flying saucers will have to pay for themselves. "If you work for me, you > better understand that," he says.

While I'm all for commercialization. I think a lot of game changing technology takes a while to develop before they can be commercialized.

But google having too much money can be a curse as well. Best innovation comes when you think like a cockroach to survive.

</comment><comment><USER65> > "No one wants to face the reality that this is an advertising company with a > bunch of hobbies."

great line.

> Many former X employees blame overexuberance on the part of Google's > marketing division for the hostile reception that greeted Google Glass

it was $3000, hard to get, and what were people really going to do with it?

Sure. Marketing.

> He notes that although Glass was marketed to the public as the Explorer > Edition, many people assumed it was a finished product.

Fair point, but how many years are people supposed to eagerly wait?

> The effort, known internally as Tableau and championed by Brin, had been a > plan to create gigantic TV screens.

what? lol.

<comment><USER66> When I worked at Yahoo (2003-2005) the talk was the same: Engineering, which dominated the company, largely thought Yahoo was a tech company.

But Yahoo was an advertising and media company.

Yahoo never quite managed to resolve that conflict, which rose straight to the top, and it kept causing stupid things like the massive amount of money that was poured into their own search engine instead of focusing on the content and services that actually had users continue to come back and stay within the network (e.g. Mail never made much money, but a large portion of users went to the Yahoo homepages to log into their e-mail, and a good portion would end up clicking on various news articles etc.)

Lots and loss of engineering effort went into project that'd never have been greenlit if Yahoo had accepted earlier what it was.

But Yahoo had a great deal of Google-envy at the point I was there, and it coloured a lot of decisions. At the same time, Yahoo _didn 't_ have the cash to operate like Google. E.g. where Google engineers could order servers by the dozens without justifying it, I had to present my arguments to a harware review group in Sunnyvale (I ran a team in Europe) to get a single new server to run the European billing system that processed millions of dollars worths of payments. So on one hand they wanted to be more like Google, on the other hand they didn't want to pay or it, and meanwhile what made Yahoo money was first and foremost the content side.

</comment></comment><comment><USER67> I feel like Google is still stuck in 2005. Not much has changed from top of my head.

All of their products are still based on their end-user being the product since everything is free and people expect it to be that way.

I don't see anything changing because they are literally printing money from their monopoly on online advertising. They have no serious competition and no, Facebook is not really great for targeted advertising unless you are a big brand.

Without any real external pressure as a result of their own success, it's beginning to look a lot like Microsoft of 2005.

Even from a developer's point of view, Amazon has usurped the cloud space with Azure now playing catchup and winning.

Much as Satella realized that Microsoft needs to stop being like Google and more B2B like Oracle, Google needs to find it's new place.

Vast majority of the public is unlikely to give a shit about their privacy in return for free productivity and entertainment but demographics change.

<comment><USER68> "All of their products are still based on their end-user being the product since everything is free" Counterexamples: GSuite (fka Google Apps) Google Pixel Google Home Google Wifi Google Cloud Platform

</comment></comment><comment><USER69> It's funny to me that investors and biz press are surprised that Brin & Page meant what they said in the 2004 letter.

<comment><USER70> Ideals like that are the sort of thing you expect companies to abandon after a few years. And the idealism has been draining from google for years. It barely exists anymore beyond internal-facing perks and their cult-ish belief that their shit doesn't stink.

That letter also included the famous "don't be evil." Does anyone really think that in a few hundred years time, people aren't going to look back and laugh at the feeble rationalizations that have currently taken hold to convince otherwise clued-in people that the most pervasive surveillance apparatus in the world isn't at least a little bit evil?

</comment></comment><comment><USER65> > Bill Maris, the CEO of its venture capital arm, GV

isn't google ventures doing pretty well though?

<comment><USER71> You might find this an interesting read: [http://www.recode.net/2016/12/5/13842608/former-google- ventu...](http://www.recode.net/2016/12/5/13842608/former-google-ventures-ceo- raising-new-230-million-fund-focus-health-care)

</comment></comment><comment><USER72> Except in 2008, during a developing worldwide financial crisis, when the new CFO Patrick Pichette started figuring how much was actually getting spent on things such as cafeterias and childcare subsidies (for the onsite service).

So, yeah, never.

</comment><comment><USER73> Huh. Is "Alphabet" a pun? The "other bets" lose money, while the "alpha bet" (Google) makes lots.</comment>


Ask HN: How to find a project we enjoy again? - philipp-spiess

My best friend and I have been working on different projects together for more than 8 years now. Our last project was fun, but we didn't reach our goal (1 paying customer in 3 months), so we decided to quit it. Now we are both enthusiastic to work on another project, just for the sake of fun, but somehow we have become very critical and don't like any of our ideas.<p>What should we do? How can we find a project that we can enjoy again? <comment><AUTHOR> This may be relevant to your situation: [https://www.physics.ohio- state.edu/~kilcup/262/feynman.html](https://www.physics.ohio- state.edu/~kilcup/262/feynman.html)

<comment><USER2> Do I get it right? Your point is: we should just build anything, no matter if we think it's valuable?

<comment><AUTHOR> Yeah. The idea is just to do something _you_ think is awesome first because that's the hard part.</comment></comment></comment>


Ask HN: Random ideas for Raspberry pi - <AUTHOR>

I am taking off almost 15 days off from work and can't travel or do anything outdoor so, planning to tinker with my raspberry pi. Suggest me some ideas that i can do with it. I have seen few interesting things people did but curious to know more.. <comment><USER2> Grab an SDR and start hacking. [http://www.rtl-sdr.com/](http://www.rtl- sdr.com/)

I currently have 6 Pi's in my home and office with SDRs, each grabbing interesting stuff off the public airwaves.

</comment><comment><USER3> Using the Pi for home automation tasks is always fun?

maybe create a handheld portable game emulator?</comment>


Don't be a Human Keyboard - <AUTHOR> http://sysadvent.blogspot.com/2010/12/day-13-dont-be-human-keyboard.html

<comment><USER2> "Humans are creatures of habit. If you reinforce a behavior, it will persist, and even spread to othercoworkers or users. You don't want to become the company-wide interface to answering the "is mysql down" question."

I think that was well said.

I believe it's one of the keys to be able to have fun working in a big company.

</comment><comment><USER3> I don't think things are as simple as he states. The ideal company with an intranet that is easily browsable/searchable, completely documents all procedures and remains up-to-date may exist, but I haven't seen it.

Because of that, 'bothering' collleagues with (in your eyes) simple questions can be an essential way of keeping the knowledge of your employees up-to-date ('Is mySQL down?' => 'do you know about the new SMS feature in our dashboard?'

Because of that, I would both give colleagues who ask simple questions fish and teach them where I got it from.

But I guess the optimum strategy depends heavily on one's personality and the environment one works in. For example, in some environments, politely answering every "where is the any key?" question, but reporting to your boss that those questions take a day a week might be the best way to handle this.

</comment><comment><USER4> This is what made me a heavy user of lmgtfy.com in my daily office routine.

That small amount of shame when receiving a lmgtfy-link from me in response to a question greatly helped reducing the amount of stupid questions I get asked.

</comment><comment><USER5> I fight against this all the time.

The bad thing is that I keep freeing up too much time and that can create its own problems.</comment>


Show HN: Perfect 2.0 - Send a postcard with ease - <AUTHOR> https://www.postperfect.co/#home

<comment><USER2> Before filling out the postcard I would want to know how much it costs to send it and what pictures I can put on the front. Neither of those seem possible</comment>