Why Google Stores Billions of Lines of Code in a Single Repository (2016) - <AUTHOR> https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext <comment><USER3> I feel terrible for anyone who sees this and thinks, "ah! I should move to a monorepo!" I've seen it several times, and the thing they all seem to overlook is that Google has THOUSANDS of hours of effort put into the tooling for their monorepo. Slapping lots of projects into a single git repo without investing in tooling will not be a pleasant experience.

<comment><USER4> We moved to a monorepo about 2 years ago and it has been nothing but success for us.

We have quite a few projects but only 4 major applications. Maybe it is that a few of our projects intertwine a bit so making spanning changes in separate repositories was a pain. Doing separate PRs, etc. Now changes are more atomic. Our entire infrastructure can be brought up in development with a single docker-compose file and all development apps are communicating with each other. I don't think we've had any issues that I can recall.

We are a reasonably small team though, so maybe that is part of it.

<comment><USER5> My previous had a monorepo for the website and backend (but not the mobile apps) which was insane to work with (as a coder I had a dedicated 128 core box to work on, some engineers had more than one, less intense engineers shared one) and a substantial amount of my time was spent just finding code. I guess most engineers just end up working in some nook and so that searching code constantly becomes less of an issue (it never did for me) but the code / debug cycle was dreadful.

I should add that a huge amount was invested in tooling. We had an in-house IDE with debug tools that could step through serverside code. We had a highly optimized code search tool. We had modified a major version control system so it could handle our codebase. (Indeed we picked our version control system because we needed to fork it and the other major version control system was less amenable to our PRs.)

My current job we have a micro service architecture and lots of small, focused repos. Each repo is self-documented. Anyone can checkout and build anything. We don't need obscene dev servers. We have not hugely invested in tools or workflow.

Client apps are unavoidably larger repos than the services apps.

Based on my personal experience, I think monorepos are nuts.

<comment><USER6> How is this

       company
         /ProjectA
           /.git
         /ProjectB
           /.git

easier to browse than this?

       company
         /.git
         /ProjectA
         /ProjectB

You still need to find the project A repo if you don't use monorepos. And even if you do use monorepos everything doesn't have to be one monoloithic build hogging down your IDE, you can still have microservices with the code for each hosted in the same repo. You seem to conflate monorepo with lots of other things.

<comment><USER7> In the former each project is a self contained unit and if I'm working on project B I can forget project A even exists, which is lovely caused I've got enough to deal with on B as is. Each project can be branched individually, the log of project B is not polluted with commits to project A, I can rebase and not get a bunch of commits I don't care about.

The later forces me to be aware of the entire universe in that repo.

<comment><USER6> In the later you can also forget project A exist. git log works in subdirectories out of the box and if you didn't touch any files in project A a rebase will be trivial without any merge conflicts. Branching is also free (unless you are still in the SVN stoneage that requires a copy of each file) so it doesn't matter if you branch the whole monorepo or just a single project.

<comment><USER8> > unless you are still in the SVN stoneage that requires a copy of each file

Maybe you are thinking of CVS? In SVN, creating branches has always been cheap in both space and time.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> Is it just me, or are a lot of people here conflating source control management and dependency management? The two don't have to be combined. For example, if you have Python Project X that depends on Python Project Y, you can either have them A) in different scm repos, with a requirements.txt link to a server that hosts the wheel artifact, B) have them in the same repo and refer to each other from source, or C) have them in the same repository, but still have Project X list its dependency of project Y in a requirements.txt file at a particular version. With the last option, you get the benefit of mono-repo tooling (easier search, versioning, etc) but you can control your own dependencies if you want.

edit: I do have one question though, does googles internal tool handle permissions on a granular basis?

<comment><USER10> Single repo is one design that coherently addresses source control management and dependency management.

The key is to let the repo be a single comprehensive source of data for building arbitrary artifacts.

<comment><USER11> A single repo makes it a bit tricky to use some library in version A for project X and version B for project Y.

<comment><USER12> I think that's actually a good thing. Allowing different projects to use different versions of a 3rd-party package may be convenient for developers in the short term, but it creates bigger problems in the long term.

<comment><USER11> It depends on the industry. In some places changing a dependency, no matter how trivial the change, entails _a lot_ of work. Think for example about embedded systems where deploying is a lot harder than pushing a Docker image somewhere. It is often far cheaper to analyze whether the fixed bug can be triggered to avoid upgrading unless necessary.

<comment><USER12> In those situations, why not go ahead and keep the code up-to-date and consistent, and simply not deploy when you don't need to?

<comment><USER11> Because that costs money now that could be spent on something that actually produces a profit.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> Well, it's not so uncommon. For instance, OpenBSD, NetBSD repos are sort of monolithic. And, believe it or not, there are some advantages. For instance, let's take a look at OpenBSD 5.5 [0] release notes:

> OpenBSD is year 2038 ready and will run well

> beyond Tue Jan 19 03:14:07 2038 UTC

OpenBSD 5.5 was released on May 1, 2014. While Linux is still "not quite there yet" y2038-wise. y2038 is a very complex issue, while it may look simple - time_t and clock_t should be 64-bit. This requires changes both on the kernel -- new sys-calls interfaces [stat()], new structures layouts [struct stat], new sizeof()-s, etc. -- and the user space sides. This, basically, means ABI breakage: newer kernels will not be able to run older user space binaries. So how did OpenBSD handle that? The reason why y2038 problem looked so simple to OpenBSD was a "monolithic repository". It's a self-contained system, with the kernel and user space built together out of a single repository. OpenBSD folks changed both user space and kernel space in "one shot".

IOW, a monolithic repository makes some things easier:

a) make a dramatic change to A

b) rebuild the world

c) see what's broken, patch it

d) while there are regressions or build breakages, goto (b)

e) commit everything

[0] [http://www.openbsd.org/55.html?hn](http://www.openbsd.org/55.html?hn)

[UPDATE: fixed spelling errors... umm, some of them]

-ss

<comment><USER14> The reason why y2038 problem looked so simple to OpenBSD has little to do with "monolithic repository" and everything to do with "happy to break kernel ABI compatibility". You're saying as much yourself.

Monolithic repository might have been a tool that helped enforce it, but that's not what made it happen. It's the decision that ABI could be broken that did.

And that's also why it hasn't happened in Linux yet. Even if there was a monorepo containing all the open source and free software in the world (or at least, say, that you can find in common distros), the fact that there's a contract to never break the ABI makes it simply hard to do.

<comment><USER13> > Monolithic repository might have been a tool that helped enforce it, > but > that's not what made it happen. It's the decision that ABI could > be broken > that did.

Well, there are probably some subtle details which I'm missing, and may be you are totally right.

The way it looks to me is as follows: They are "happy to break kernel ABI compatibility" because the repository is monolithic - they break ABI, they immediately fix user space apps.

E.g. NetBSD time_t 64-bit commit: [https://marc.info/?l=openbsd- cvs&m=137637321205010&w=2](https://marc.info/?l=openbsd- cvs&m=137637321205010&w=2)

They patched the kernel:

    	 sys/kern       : kern_clock.c kern_descrip.c kern_event.c
    	                 kern_exit.c kern_resource.c kern_subr.c
    	                 kern_synch.c kern_time.c sys_generic.c
    	                 syscalls.conf syscalls.master vfs_getcwd.c
    	                 vfs_syscalls.c vfs_vops.c

and fixed broken user space at the same time:

...

    	 sys/msdosfs    : msdosfs_vnops.c
    	 sys/netinet6   : in6.c nd6.c
    	 sys/nfs        : nfs_serv.c nfs_subs.c nfs_vnops.c xdr_subs.h
    	 sys/ntfs       : ntfs_vnops.c
    	 sys/sys        : _time.h _types.h dirent.h event.h resource.h
    	                 shm.h siginfo.h stat.h sysctl.h time.h types.h
    	                 vnode.h
    	sys/ufs/ext2fs : ext2fs_lookup.c
    	sys/ufs/ufs    : ufs_vnops.c

...

There is no "transitional" stage, when the kernel is already patched, but no user space apps are ready for those changes yet. It all happens at once.

-ss

<comment><USER7> > There is no "transitional" stage, when the kernel is already patched, but no > user space apps are ready for those changes yet. It all happens at once.

What about third party apps? It's not a fully self contained system, there are binaries out there running on openBSD that the openBSD devs have never heard of, and they were broken by the change.

<comment><USER15> BSDs simply don't guarantee ABI stability, so no third party app should ever make a syscall directly. It all goes via libc. So, yes, from that perspective, it is a fully self-contained system.

In practice, third-party apps sometimes think that they know better, and get broken. Anything written in Go, for example:

[https://github.com/golang/go/issues/16272](https://github.com/golang/go/issues/16272)

<comment><USER16> Sure they do guarantee the ABI stability (within major release), but the main thing here is that FreeBSD - like pretty much any other operating system, but differently from Linux - maintains the stability at the libc level, not at syscall level.

<comment><USER15> I meant kernel ABI specifically, of course.

And it's a guarantee that is, essentially, useless for any purpose other than the interaction between the base system and the kernel - i.e. not for third party software.

</comment></comment></comment></comment></comment></comment></comment><comment><USER17> Managing dependencies and versions across repos is a pain. Refactoring across repos is quite hard when your code spreads across repos considering the tree of dependencies.

Unfortunately Git checkout all the code, including history, at once and it does not scale to big codebases.

The approach that Facebook chose with Mercurial seems a good compromise ( [https://code.fb.com/core-data/scaling-mercurial-at- facebook/](https://code.fb.com/core-data/scaling-mercurial-at-facebook/) )

<comment><USER18> Git works very well when the code is _distributed_. Which funnily enough is in the name. That we are using git as a centralized repository is a case of "Why do I need a screwdriver when I have a hammer?".

<comment><USER19> There's nothing about git that requires you to use it like the kernel does. Centralized version control is just a special case of decentralized, if you're using git. You still get the benefits of your repo being a peer of the master repo, like local branches.

</comment></comment></comment><comment><USER20> Maybe I'm not cool enough to understand this, but I don't see the draw for monorepos. Imagine if you're a tool owner, and you want to make a change that presents significant improvements for 99.9% of people, but causes significant problems for 0.1% of your users. In a versioned world, you can release your change as a new version, and allow your users to self-select if/when/how they want to migrate to the new version. But in a monorepo, you have to either trample over the 0.1%, or let the 0.1% hold everyone else hostage.

Conversely, imagine if you're using some tools developed by a far off team within the company. Every time the tooling team decides to make a change, it will immediately and irrevocably propagate into your stack, whether you like it or not.

If you were at a startup and had a production critical project, would you hardcode specific versions for all your dependencies, and carefully test everything before moving to newer versions? Or would you just set everything to LATEST and hope that none of your dependencies decide to break you the next day? Working with a monorepo is essentially like the latter.

<comment><USER21> I've worked both at Google (only as an intern, though) and at other very very big companies with gargantuan code bases. At that scale, with software that is constantly in flux, pretty much the last thing you want is having to keep compatibility between several versions of a component. It's bad enough if you have to do it for external reasons, but if the only reason is so that "others in the company have a choice" then... no, just no.

You might think this ought to be trivial by having clear API contracts, but that's a) not how things work in practice if all code is effectively owned by the same, overarching entity and, more importantly, b) now you have an enormous effort to transition between incompatible API revisions instead of just being able to do lockstep changes, for no real gain.

Even if you manage to pull that off (again, for what benefit?), it will bite you that 1.324.2564 behaves subtly different from 1.324.5234 even though the intent was just to add a new option and they otherwise _ought_ to have no extensional changes in behavior.

<comment><USER20> It took me a while to figure out that you're disagreeing with me, because your last paragraph is a perfect example of why monorepos are so dangerous.

Imagine a tooling team on a different continent that makes some changes this afternoon. Like you said, their intent is just to add a new option, and it ought to have no extensional changes in behavior, but it still ends up behaving subtly different. The next morning, all your services end up broken as a result.

In a versioned world, you can still freeze your dependency at 1.324.5234, and migrate only when you want to, and when you're feeling confident about it.

In a monorepo world, you don't have a choice. You've been forcefully migrated as soon as the tooling team decides to make the change on their end. They had the best of intentions, but that doesn't always translate to a good outcome.

FWIW, I'm currently working at a large famous company that uses a monorepo. Color me not-impressed. I do think that having a single repository for an entire team/project is a good idea. Hundreds of different projects and teams who've never seen one another? Not so much.

<comment><USER21> > In a versioned world, you can still freeze your dependency at 1.324.5234, > and migrate only when you want to, and when you're feeling confident about > it.

The correct course of action is to either reverse/fix the code change to the library you depend on, or if your code is clearly using the library wrong and can be easily fixed, to do that. Not to let the whole ecosystem slowly spiral out of control.

Either way, the point is that it will force the issue to be resolved, quickly, and the code base to move forward.

The tools/libraries you depend on are themselves dependent on other libraries and tools. They may have done changes that are necessary to continue working, which you are not picking up if you stay behind. They will do IPC and RPC and always rely on their infrastructure being current.

>In a monorepo world, you don't have a choice. You've been forcefully migrated

Yes, and that's good, because:

> and migrate only when you want to, and when you're feeling confident about > it.

... does not help in moving the code forward.

If your change will break others, you need to coordinate with those others so that the transition happens gracefully, not let them live on what amounts to unsupported (and slowly more incompatible) code.

<comment><USER20> _> > In a monorepo world, you don't have a choice. You've been forcefully migrated

> Yes, and that's good_

In your projects, have you configured your build system to always auto-pull the latest version of every single dependency you have? If not, you're not practicing what you've claimed above.

FWIW, java-maven used to allow specifying LATEST/RELEASE versions, so that the latest version will always be auto-pulled on every build. They later removed that option entirely, because they realized how dangerous that is.

[https://stackoverflow.com/questions/30571/how-do-i-tell- mave...](https://stackoverflow.com/questions/30571/how-do-i-tell-maven-to-use- the-latest-version-of-a-dependency)

</comment></comment></comment></comment></comment><comment><USER22> This is clearly detrimental to external projects such as Go packaging, since _their own_ developers will never be looking at dependency problems in the same way as outside groups.

Monorepo also bugs me because there will _always_ be some external package you need, and invariably it's almost impossible to integrate due to years of colleagues making internal-only things assume everything imaginable about the structure and behavior of the monorepo. There will be problems not handled, etc. and it leads to a lot of NIH development because it's almost easier in the end.

Also, it just feels _risky_ from an engineering perspective: if your repository or tools have _any_ upper limits, it seems like you will inevitably find them with a humongous repo. And that will be Break The Company Day because your _entire_ process is essentially set up for monorepo and no one will have any idea how to work without it.

<comment><USER23> What about Android and 800-1,000 git repos?!

Have seen the pain trying to manage that across larger teams (e.g. thousands of devs) - and no the "repo" tool is not sufficient.

<comment><USER24> I'm very curious, what pain did you see with the repo.py tool?

</comment></comment></comment><comment><USER25> Having worked at different companies adopting both monorepo and the multiple repos approach, I find monorepo a better normalizer at scale in consolidating all "software" that runs the company.

Just like what many commenters here have mentioned, the monorepo approach is a forcing function on keeping compatibility issues at bay.

What you don't want is to end up in a situation where teams reinvent their own wheels instead of building on top of existing code, and at scale, I think the multiple repo approach tends to breed such codebase smell. [1] I'm sure 8000 repos is living hell for most organizations.

[1] - [https://www.youtube.com/watch?v=kb-m2fasdDY](https://www.youtube.com/watch?v=kb-m2fasdDY)

<comment><USER26> I really liked that talk! Lots of relevant information and I can definitely relate, working at a Amazon. Wouldn't say that we are hurt by all of the same problems (we have solutions that work very well for some of them), but we definitely are aware of them.

</comment></comment><comment><USER27> One of my former managers had worked a long time at Google and was present for the advent of Google's in-house tooling developed around their monorepo.

His account was that it was basically accidental, at first resulting from short term fire drills, and then creating a snowball effect where the momentum of keeping things in the Perforce monorepo and building tooling around it just happened to be the local optimum, and nobody was interested in slowing down or assessing a better way.

He personally thought working with the monorepo was horrible, and in the company where I worked with him, we had dozens of isolated project repos in Git, and used packaging to deploy dependencies. His view, at least, was that the development experience and reliability of this approach was vastly better than Google's approach, which practically _required_ hiring amazing candidates just to have a hope of a smooth development experience for everyone else.

I laugh cynically to myself about this any time I ever hear anyone comment as if Google's monorepo or tooling are models of success. It was an accidental, path-dependent kludge on top of Perforce, and there is really no reason to believe it's a good idea, certainly not the mere fact that Google uses this approach.

<comment><USER28> Do you wonder whether he is a reliable narrator?

<comment><USER27> I don't, but it's fair to ask. He was unequivocally the best senior manager I've worked with. Extremely technically smart but skilled at letting people under him work autonomously, good communicator, cared a lot about pushing best practices past bureaucratic barriers.

His description of Google made it seem like it had the same dysfunction every place has. And the monorepo was a totally mundane, garden variety eyesore kind of in-house framework that you'll find anywhere.

I think he recognized the usefulness of just working with it and picking battles. He was just dumbfounded that any outsider would see the monorepo project and think it possibly had any relevance for anyone else. It was just a Google-history-specific frankenstein sort of thing that got wrangled with tooling later. The supposed benefits are all just retrofitted on.

</comment></comment></comment><comment><USER29> Google's handling of their source code makes me wanna work there.

I don't like distributed version control systems with hundreds of repositories spread out. It makes management more complicated. I understand this is a minority view, but that is my experience. It was easier to work in a single Perforce repository than hundreds of Git or Mercurial repos.

<comment><USER30> Distributed vs. centralized VCS has very little directly to do with many vs. monolithic repos. After all, git was originally developed for a project with a large monolithic repo. Distributed VCS and many small repos got popular around the same time, but that's partly coincidental (microservice architectures getting popular, npm community preferring extremely small libraries) and partly because of GitHub making it very cheap in money/time to have many git repos.

</comment></comment><comment> a-dub It should be noted that the monolithic model is somewhat encouraged by the client mapping system in Perforce, which was Google's first version control system so it is unclear to me if this was deliberate or just a side effect of the best VCS of the time.

I also still have doubts around the value of a monorepo, in the article they claim it's valuable because you get:

Unified versioning, one source of truth;

Extensive code sharing and reuse;

Simplified dependency management;

Atomic changes;

Large-scale refactoring;

Collaboration across teams;

Flexible team boundaries and code ownership; and

Code visibility and clear tree structure providing implicit team namespacing.

With the exception of the niceness of atomic changes for large scale refactoring, I don't really see how the rest are better supported by throwing everything into one, rather than having a bunch of little repos and a little custom tooling to keep them in sync.

<comment><USER31> Incrementally monolithic CL number is also useful. You can mark quite a lot of things with it - not only binary releases, but other developments too (configuration files, etc.). At the end your binary "version" comprises of main base CL + cherrypicked individual CL's - rather than branch with these fixes - I guess one can encode this too with git/hg - by using sha hashes - but this becomes much bigger in terms of information, and human handling it.

I guess not very strong point, but using CL numbers (I'm working with perforce mostly these days) makes things easier. And having one CL monothonically increasing all over all source code you have even better - you can even reference things easier - just type cl/123456 - and your browser can turn it into a link. Among many other not so obious benefits...

<comment><USER32> Most popular Git frontends (GitHub and GitLab too, I believe) let you link to commits with just the first 5-6 characters of the hash. I don't think that's much different to remember than a Perfore CL number.

<comment><USER31> To me the issue is when mentally trying to work with these numbers, P4 & G4's numbers increment, so I can tell which one came before the other - I can't do this with hashes. I'm sure I can get used to the other way, but this cannot easily be ignored.

<comment><USER33> Mercurial gives you those incrementing numbers through its revlog, and they map to revisions, so you get that facility there.

</comment></comment></comment></comment></comment><comment><USER34> Previous thread:

[https://news.ycombinator.com/item?id=11991479](https://news.ycombinator.com/item?id=11991479)

</comment><comment><USER35> > Google's monolithic software repository, which is used by 95% of its > software developers worldwide, meets the definition of an ultra-large-scale4 > system, providing evidence the single-source repository model can be scaled > successfully

This 95% number is the most surprising part of the article. That implies that the sum of engineers working on Android + Chrome + ChromeOS + all the Google X stuff + long tail of smaller non-google3 projects (Chromecast, etc) constitute only 5% of their engineers. Is e.g. Android really that small?

<comment><USER36> They must have meant that 95% of Google engineers use the monorepo in some capacity, even if the majority of their work is done in a different repo.

</comment></comment><comment><USER37> My company has a 50m LOC Python codebase in a monorepo. It works really well, given the rate of change of thousands of developers globally. That is only possible because of the significant investment in devtools, testing and the deployment infrastructure.

Here is "Python at Massive Scale", my talk about it at PyData London earlier this year:

[https://youtu.be/ZYD9yyMh9Hk](https://youtu.be/ZYD9yyMh9Hk)

</comment><comment><USER38> We work with an monorepo since Septemeber 2017. I wrote about the migration:

[https://timkrueger.me/a-maven-git-monorepo/](https://timkrueger.me/a-maven- git-monorepo/)

Our developers like it, because they can use 'mkdir' to create a new component, search threw the complete codebase with 'grep' and navigate with 'cd'.

</comment><comment><USER39> I wish more developers knew of the wonderful "repo" tool[0] developed by the Android devs which allows a monorepo _perspective_ of many git repositories. Breakdown of the repo tool and example manifest files [http://blog.udinic.com/2014/05/24/aosp-part-1-get-the- code-u...](http://blog.udinic.com/2014/05/24/aosp-part-1-get-the-code-using- the-manifest-and-repo/)

[0] [https://source.android.com/setup/develop/repo](https://source.android.com/setup/develop/repo)

</comment><comment><USER40> > includes approximately one billion files

...

> including approximately two billion lines of code

_also_

> in nine million unique source files

I should insert a joke about how well the system would do if each source file contained more than two lines of code.

But seriously, this summary could use some work.

<comment><USER41> Binary files (arbitrary example: images used for golden screenshots in tests) have no line counts and are likely skewing the numbers here -- in the way you're (logically) looking to interpret them at least.

From a system design perspective, being able to handle a large number of files regardless of type is an interesting challenge, as is being able to handle a large number of highly indexed text files. All three of those statistics seem potentially interesting for different audiences that might read this paper.

</comment></comment><comment><USER42> It's not just devops that you need to pull off a large monorepo; the other big thing is a strong testing culture. You have to be able to rely on unit tests from across the code base being a sufficient indicator of whether your commit is good. AND a presubmit process that can compute which parts of the monorepo get affected by your diff, and run tests against them automatically before committing your diff.

Google not only has the above but also has a strong pre-submission code review process which catches large classes of bugs in advance.

</comment><comment><USER31> Here is the video (with Rachel Potvin), predating the article by some months: [https://www.youtube.com/watch?v=W71BTkUbdqE](https://www.youtube.com/watch?v=W71BTkUbdqE)

</comment><comment><USER43> I've used monorepo for few small related projects and it worked just fine for me. Much easier to make related changes across several projects.

</comment><comment><USER44> This is probably a stupid question, but I couldn't find an answer. Does this mean Google keeps all of its different products in all their different languages and environments in one repo? So like, Android lives in the same repo as Gmail, which is the same repo as all the Waymo code and the Google search engine code as well? That seems insane to me.

<comment><USER45> Android & chromium are kept outside the monorepo

</comment></comment><comment><USER46> Version controlled repositories are like business offices.

You can have your entire company in one location, or the entire company in separate locations. The _most_ important thing is the logical rather than physical organization: team structure, executive leadership, inter-org dependencies, etc. You can achieve autonomy and good structure with or without separate locations.

A single location reduces barriers, but at some point multiple locations can solve physical and logistical challenges. General rule of thumb is to own and operate office space in a few locations as possible, but at some point you have to take drastic measures one way or another.

(Notice that Google had to invent their own proprietary version control system just for their monorepo. And not even Google _actually_ uses a single repo as the source of truth: e.g. Chromium and Android.)

</comment><comment><USER47> Im sure properly organized it's okay, but from what I've seen it's mediocre at best, especially with legacy/technical debt it's a huge mistake.

Start breaking that repo apart, because it probably isn't very/hopefully depending on the debt that exists.

<comment><USER48> One of the big advantages of the monorepo is actually that it prevents technical debt from accumulating. If a change somewhere else breaks your code, you can't put off dealing with it -- you are forced to fix the issue immediately.

<comment><USER49> Tech debt is a useful tool and we shouldn't have zero tolerance. I can see wanting to deprecate old versions promptly, but I can't see _instantly_ deprecating _every_ old version with no workaround for mitigating emergencies.

</comment></comment></comment><comment><USER50> Seems like Google uses its own custom Source Control & tools - [https://www.quora.com/What-version-control-system-does- Googl...](https://www.quora.com/What-version-control-system-does-Google-use- and-why).

</comment><comment><USER51> [https://en.wikipedia.org/wiki/Conway%27s_law](https://en.wikipedia.org/wiki/Conway%27s_law)

> "organizations which design systems ... are constrained to produce designs > which are copies of the communication structures of these organizations."

Interestingly, in light of the above adage, this massive repo is organized (if that's the word for it) like a bazaar or flea market. (Rather than like a phone book [https://en.wikipedia.org/wiki/Yellow_pages](https://en.wikipedia.org/wiki/Yellow_pages) )

</comment><comment><USER52> > Trunk-based development. ... is beneficial in part because it avoids the > painful merges that often occur when it is time to reconcile long-lived > branches. Development on branches is unusual and not well supported at > Google, though branches are typically used for releases.

This sounds like the SVN model to me where branches are cumbersome and therefore they are very rare. After getting used to the Git branching model where branches are free and merges are painless, it would be very hard to go back to the old development model without branches.

</comment><comment><USER53> How does CI work with a monorepo? Do you always have to run all the tests and build all the artifacts? Or are there nice ways to say "just build this part of the repo"?

<comment><USER54> You specify targets. Just like using bazel: bazel build //tensorflow/blah/....

I maintain a small part of the monorepo, and it's really nice to be able say "Run every test that transitively depends on numpy with my uncommitted changes", so you can know if your changes break anybody who uses numpy when you update the version.

Personally I think it would be neat if there was an external "virtual monorepo" that integrated as-close-to-head of all software projects (starting at the root, that's things like absl and icu, with the tails being complex projects like tensorflow), and constantly ran CI to update the base versions of things. Every time I move to the open source world, I basically have to recompile the world from scratch and it's a ton of work.

</comment></comment><comment><USER55> I have slight experience with both monorepos and smaller repos and I think they can both work. The advantage of smaller repos is that it forces different components to expose well designed API's. Bigger repos make sense for products and embedded software, smaller repos make sense for platforms build up of small services communicating on the internet.

<comment><USER30> Smaller repos force different components to expose APIs, but I don't think it forces or even encourages the APIs to be well designed. In some cases, having work spread across multiple repos can impede iterative development, meaning that you risk half-assed or, uh, two-and-a-half-assed implementations.

Also, when someone's asking for review for a change that encompasses, say, a change to a service, a change to a client library for that service, and a change to 2-3 other services that use that client library, I know that I cringe a little when suggesting a change, knowing that to implement it is going to require a commit on all of these different repos, waiting for CI to run on each one, etc. I try to only use that impulse to counter the urge to bikeshed, but the temptation is there.

</comment></comment><comment><USER56> Is this really relevant for anyone except for "google scale" companies? For most teams, managing 30-40 services backed by git repos isn't a huge task and doesn't cause many problems.

Is there mature tooling that helps teams manage this, or is this proprietary google magic tooling?

<comment><USER57> Most teams can probably get by with much fewer than 30-40 services. Unless you have 30-40 groups within your team.

<comment><USER56> Even if they had that, managing the contract between and in a small group isn't super difficult.

</comment></comment></comment><comment><USER58> I don't understand why gitlab/github/bitbucket don't provide better tools for monorepo. This is a topic pretty trendy. But there is absolutely no tools helping with control access, good ci, ...

<comment><USER31> What's missing in these is cross-reference, which is not possible without somewhat established BUILD system (caps "pun-intened") - e.g. like bazel/build, then a source code indexer, etc, etc.

This becomes very critical for doing reviews, since it allows you to "trace" things without running them, apart from many other things. For example large scale refactorings looking for usages of functions, and other examples like it.

Why githab/gitlab/etc. can't do it? Well because hardly there could be one encompassing BUILD system to generate correctly this index.

<comment><USER58> They can create a standard file format that has to be generated by build system. github is in a pretty powerful position. They can create even a shitty version of it and people will follow.

I've been thinking about a tool like this for a long time. A way to attach to each commit not only the diff in the code, but also the list of places affected by the changes (usages of functions that are modified for example). Then during review we wouldn't have only a stupid diff. We would have a list of place to check to be sure that the changes make sense in the context of the project.

<comment><USER31> Even if they can, it's one thing indexing your own source files every night, another indexing a much bigger amount + massive amounts of branches, clones, etc. (I'm talking about github) - e.g. not practical - as there is no no clear way to say which branch (from git) must be indexed (obviously not all) - e.g. there is no encompassing "standard" saying so.

That by itself is another BIG PLUS for mono-repo (and "mono"-rules) - <USER2> are done one (opinionated) way, trunk based development - but thus giving you things that you won't be able to have normally.

Now indexing source file is not an easy and cheap task - it's basically a huge MapReduce done over several hours (just guessing), so there must be a reason for this to be done.

</comment></comment></comment></comment><comment><USER59> The giant monorepo works only if you're using SVN, with Git it would be tremendous

<comment><USER60> Unless you change git like Microsoft did: [https://blogs.msdn.microsoft.com/bharry/2017/05/24/the- large...](https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git- repo-on-the-planet/)

<comment><USER61> sure, but GitHub / GitLab / Bitbucket don't offer it

</comment></comment></comment><comment><USER62> Sorry, but as someone who has been in orgs that do both, mono repo is a mistake. Constant needs to pull unrelated changes before pushing, pipelines requiring to grab the whole repo for dependencies, etc. I understand the arguments for mono repo, but never think it's nothing that outweighs the cons.

<comment><USER23> Well those are issues around having a git mono repo - where the repo is the unit of change - you get it or you don't.

With mono repos such as SVN or Perforce you just work on whatever subset you want.

</comment></comment><comment><USER63> I love these articles. Is there a wiki or collection of detailed descriptions of large company tech practices that isn't marketing blargh.

I read years ago about Google data ingest, locator process but neglected to bookmark so now can't find the reference.

<comment><USER64> Me too. I don't know of a collection, but others can be found at [https://ai.google/research/pubs/](https://ai.google/research/pubs/) [https://research.fb.com/publications/](https://research.fb.com/publications/) [https://www.microsoft.com/en- us/research/search/?q&content-t...](https://www.microsoft.com/en- us/research/search/?q&content-type=publications) and similar (though only a small fraction give hints about at scale practices, and those would be neat to collect in one place).

Closely related to this post: just noticed a 2018 case study on Advantages and Disadvantages of a Monolithic Repository [https://ai.google/research/pubs/pub47040](https://ai.google/research/pubs/pub47040)

</comment></comment><comment><USER65> Should probably have a [2016] tag.

<comment><USER66> Indeed, but to be fair, the information in the article is based on several research papers from 2011 [1].

And I am 100% sure the idea of having a monolithic project is several years older than that.

I am grateful that the article is re-posted in multiple websites, because just the other day I was in an interview and, while doing my coding challenge, overheard the conversation of a young computer science graduate and another interviewer. The interviewer asked him to explain what was a monolithic repository and the benefits. This guy had no idea what the interviewer was talking about and right there I realized that what many of us take for granted terminology-wise in the IT world, will certainly be a foreign language to young students who are just entering the work force.

[1] [http://info.perforce.com/rs/perforce/images/GoogleWhitePaper...](http://info.perforce.com/rs/perforce/images/GoogleWhitePaper- StillAllonOneServer-PerforceatScale.pdf)

</comment></comment><comment><USER67> (2016)

</comment><comment><USER68> A repo including configuration and data.

How about we stop considering google an engineering leader and just a search leader?

</comment><comment><USER68> A repo with configuration, secrets and data?

Can we stop considering google an engineering leader and just a search algorithm leader?

</comment><comment><USER69> I think monorepos make a lot of sense when you're talking about millions of lines of code. I'm not at all sure they make sense when you're talking about billions.

<comment><USER70> I don't think the number of linea matters. I think the interconnection of your code matters. If you have 2 sets of services that are completely uncoupled the having two monorepos for those two deployments make sense. If you can guarantee atomic changes across all services that interconnect you have the benefits monorepos give you.

<comment><USER71> Isn't this only true if you're doing full CI? Otherwise I could update my service and you can update yours to work with mine but unless we coordinate deployments you still have to worry about interface mismatches. I guess the alternative is you can just never (for a loose definition of never) make breaking changes to an interface. You can only enhance or create a new version.

</comment></comment></comment><comment><USER72> I don't care about that. For me this is incomprehensible:

Why the eff does Google have billions of lines of code in their repo?

I hope they are not counting revisions (e.g., if a single 1 million project has 100 revisions, that's 1 million, not 100 million).

I have heard that they do count generated code (so it's not all handwritten code). In that case again, I have two things to say:

\- that's a bad metric. I could overnight generate a billion lines of code with each line a printf of number_to_word of numbers from 1 to a billion. They want to measure the size of the repo? They should tell us the gigabytes, terabytes etc. But when it's lines of code, it's cheezy and childish to blow up the measure by including lines of generated code.

\- But more importantly, I hope the generated code is 90% or more of that repository. Because any less than that would mean that Google engineers have handwritten 100 million or more lines of code through out the lifetime of the company, in which case I have to ask: what bloated mess do you have on your hands? I thought you guys were the top engineers of the world.</comment>


Release: Fully operational dlclose exploit + Linux for PS4, by kR105 - <AUTHOR> http://wololo.net/2016/04/02/release-fully-operational-dlclose-exploit-linux-for-ps4-by-kr105/ <comment><USER2> The actual repo is here:

[https://github.com/kR105/PS4-playground](https://github.com/kR105/PS4-playground)

<comment><USER3> And [https://github.com/kR105/PS4-dlclose](https://github.com/kR105/PS4-dlclose)

</comment></comment><comment><USER4> So how long until Sony lawyers get the Github repo shut down?

<comment><USER5> On what grounds exactly? Sony can't forbid anyone installing other OSes on their hardware.

<comment><USER6> I don't know the boot process on a PS4, but if loading Linux requires you to modify or bypass any access controls, then it may be a circumvention violation of the DMCA.

If loading an alternate OS is a EULA violation, they might brick users machines or at least stop them from using Sony services.

<comment><USER7> But that's something the user is doing. This repo has the Linux distro, which is by itself neither a circumvention, gives instructions to circumvent, or performs a circumvention of the DRM to run, thereby passing DMCA muster.

It's somewhat similar to the situation XBMC was in back when the X still stood for Xbox -- the code itself is OK by the DMCA, but compiling it required use of unauthorized dev tools (if the user didn't have a license from MS, which they likely did not), distributing the binaries was a violation of the DMCA (due to statically linked, non-free MS libraries), and installation of said binaries was also a DMCA violation (as it required the user to circumvent the DRM on the console).

<comment><USER8> One github repo mentioned in the article is the exploit, and a basic functional webpage to "do stuff" with it.

See, [https://github.com/kR105/PS4-playground/blob/gh- pages/js/exp...](https://github.com/kR105/PS4-playground/blob/gh- pages/js/exploit.js) for example.

</comment></comment></comment></comment></comment><comment><USER9> After Xbox dev mode, I think Sony needs to be more open about their console development.

This might just be stepping stone in the right direction.

<comment><USER10> Before announcing the PS4, I was hoping Sony would adopt Linux, mainly because I think it would've done a great service to the Linux community by having games that run well on Linux, but I think it would've also helped Sony in the long run. Microsoft is already banking on "DirectX gaming" for both Windows and Xbox. Sony should've had a similar strategy. They should go all-in with Linux and Vulkan with future consoles.

<comment><USER11> Sony had their flirts with Indies and Linux and walked away from it.

The Playstation had Net Yaroze, which was almost comparable the usual devkit. A collector's item nowadays.

The PS2 had PS2 Linux, which required special RGB monitors with sync on green, but it provided almost the same access to the hardware as the devkits. The main difference was the amount of available RAM, only PS2GL and the higher level PS2 official graphics API were available. The low level one wasn't accessible.

The PS3 provided a very limited support for Linux. The graphics hardware was exposed as a plain framebuffer. Then came the firmware update that wiped Linux out of the PS3.

The PS Vita did offer a Mono based SDK, but after the initial release, they went silent. You can still access the web site, though.

The PS4 already exposes two graphics APIs much more powerful than Vulkan, with the developers being quite experienced on them.

If you search SCEE and GDC Vault web sites there are plentiful documents and sessions, NDA free, how the console market works.

<comment><USER12> There was also the PS2 version of Yabasic, which was included on the demo disc that came with the console.

If I recall correctly, this was done to avoid some kind of tax on "entertainment devices" in Europe; Sony argued that the ability to write BASIC programs made it a "computer", rather than an entertainment device.

Text could be entered using either a pad or a USB keyboard, and programs could be saved to memory cards. Some magazines even included user-submitted games on their cover disc, and the official UK magazine included a page or two of code each month which eventually built up to a playable game.

There were a few differences from the original Yabasic - gradient triangles were added, audio functionality was removed, etc. I'm told that the source code is available online somewhere, since the original Yabasic was GPL- licensed, but a quick Google search didn't turn up anything useful.

For those who are interested, a friend of mine has re-implemented the PS2 version in Javascript: [http://www.mrdictionary.net/yabasic/](http://www.mrdictionary.net/yabasic/)

<comment><USER11> Yeah, I didn't pay much attention to it beside the day I bought the PS2.

Eventually I got hold of the PS2 Linux package, but getting monitors with sync on green wasn't that easy.

</comment></comment></comment></comment></comment><comment><USER5> So can it now run Witcher 2?

<comment><USER13> I doubt it has accelerated graphics drivers.

<comment><USER5> They said it was already working in the past: [https://twitter.com/fail0verflow/status/684079550943354880](https://twitter.com/fail0verflow/status/684079550943354880)

UPDATE: just found this: [https://twitter.com/fail0verflow/status/705304480888156161](https://twitter.com/fail0verflow/status/705304480888156161)

</comment></comment></comment><comment><USER14> I wonder if this could be used to decrypt/unpack/dump/whatever the contents of games?

I know the Dark Souls community are gasping to see the cut content which may or may not live inside the Bloodborne game files.

<comment><USER15> Why would that be any easier than just ripping the content off the disc directly?

<comment><USER16> Pretty sure the game files are encrypted and all the consoles have a secure chip which decrypts them. Getting files from the game disc is easy, reading them isn't.

<comment><USER5> _> all the consoles have a secure chip which decrypts them._

I assume Steam Machines shouldn't do that and therefore they should be more mod friendly.

<comment><USER17> That has to be true for Steam Machines since you can build your own.

<comment><USER18> Especially considering they run Linux, so... I'm decently sure that Steam could easily use some form of binary protection.

</comment></comment></comment></comment></comment></comment><comment><USER19> Url changed from [http://www.theregister.co.uk/2016/04/04/full_linuxonps4_hits...](http://www.theregister.co.uk/2016/04/04/full_linuxonps4_hits_github/), which points to this.</comment>


The New York Times non-apology, and the end of lazy marketing language - <AUTHOR> http://www.elezea.com/2011/12/just-say-sorry/

<comment><USER2> Honestly, this just seems like nitpicking. Your main complaint about their email is that their apology isn't phrased in the vernacular? Don't we have better things to do with our time than complain about things like this?

I'm sure many people would think that the New York Times speaking to them in a conversational tone is inappropriate and unprofessional.

</comment><comment><USER3> There's also a big difference between "apologize for any inconvenience caused" and "for the inconvenience.."

<comment><USER4> A big difference to whom? I don't think the author intended to imply something different by using that particular language.

I doubt the author consciously thought: "i'm going to make this apology less effective," so who cares if the language didn't exactly resonate with you or the OP? I don't think it was malicious or disingenuous; if anything, the NYT needs to hire better copywriters, so their emails don't get dissected as frequently.

</comment></comment><comment><USER5> Wait, people are JUST NOW realizing that "unbeatable service!" is marketing talk?</comment>


Ask HN: Rate My Startup - LetMeGo: Let lodgings bid for your stay - <AUTHOR> http://letmego.com <comment><USER3> I'm currently employed in the hotel industry. I really have no defined role, but I perform a lot of the GM roles. I can give you a few things to think about.

First and foremost, the chain to which we belong just got out of a long fight with a large Online Travel Agency [OTA]. During this "fight", the chain really noticed that OTA's had a stranglehold on the online room distributions, especially the rates so in response, they have "initiated" some policies to nip that in the butt. (OTA's are currently killing Vegas)

I will show you the breakdown of our reservation distribution this year: 55% CHAINWEBSITE.com, 27% CHAIN Call Center, 9% GDS Travel Agencies, 9% Third Party Websites (broken down into Orbitz, Expedia and Travelocity). In short, due to our policies we CANNOT offer a lower price outside of our best available rate available on the CHAINWEBSITE.com - we could null our franchise agreement.

I only speak for one chains policies, but this chain has 6k properties and another 1k or so in development. It's not small.

Thats one thing to think about. Some more thoughts:

Getting GMs to lower rates is tough; if you give guest X rate Y, and guest Z finds out, they also want rate Y. It sometimes makes a messy situation dealing with this. Selling rooms in important, but keep rate integrity intact is also important.

Chicken/Egg conundrum. How do you plan on getting hotels to participate?

\-- In response to the above, I would personally like to search my geographical area for any travelers without having to sign the hotel up. Can I do that? Why not?

Some solutions to the above thoughts:

Instead of getting hotels to directly negotiate prices, why not use a combination of that and a GDS system like Amadeus, SABRE or Galileo. You know, so if a hotel doesn't bid on their trip to offer occupancy, then they have the option of making a booking through your system opposed defaulting to Expedia (which books through the same systems). It might solve the intermittency between the chicken/egg problem, and having an active user base - <USER2> one before the other would be essential. Overall, I don't think the hotel industry will take kindly to having to bid for guests. On the other hand, there is LOTS of money to be had and I wish you the very best. Its an ambitious start!

<comment><AUTHOR> Thanks a lot for your feedback Matt. It is very interesting and valuable. I can tell you know a lot about the industry and how it deals with OTAs.

First of all, LetMeGo is not only about low prices: it's about offers customized for each traveler depending on his/her needs (VentureBeat nailed it down here: <http://bit.ly/5vioZZ> ). Nevertheless, given that LetMeGo's pricing is "semiopaque" (the bids submitted by the lodging are ONLY visible to the traveler that submitted the itinerary), we think that some lodgings will offer prices below what they publish in CHAINWEBSITE.COM. This would be relatively similar to the way hotels do with other opaque-pricing service like Priceline and Hotwire that allow them to forget about parity. Time will tell if this assumption is correct.

By the way, we are not counting on large chains for our success. In fact, we want to work primarily with independent and small hotels, bed and breakfasts, and the large vacation rental market. Our system has been designed with them in mind first. Why? Because this area of the industry has a lot of potential that hasn't been discovered yet, and because I am against large corporations with too much power.

In reference to the chicken/egg dilemma, we have secured access to large databases of lodgings. We are inviting them all to list their lodgings for free using a very intuitive interface. They will only pay us a 10% commission per booking after they get paid.

We are staying away from the GDSs for now because most independent lodgings don't use them (outside the US independent lodgings account for more than 80% of the market). Also, because our service, as I mentioned above, is about custom offers and service, and not only about real-time and the-lowest pricing.

I had the "pleasure" of chatting with the VPs of Revenue Management of two of the largest hotel chains in the US and in the world. As you predicted in your comments, they didn't like LetMeGo's model. That, for me, was a good sign.

Thank you Matt!

<comment><USER3> Glad I could help in some way. A few other thoughts I had after reading your response.

First, don't let the VPs of Revenue Management make you deviate from targeting chains. They have THEIR profits to account for, just like each hotel has their own profits to account for. In a way, this scares them because the revenue is not coming in through THEIR channels. If you think you can help a hotel in a chain profit, if you can prove it with data and you're passionate about it then go for them too! There are some really awesome online tools I want to use, but there are just so many policies that don't allow it. Hotel industry right now is stuck between being "innovative" and "head stuck in and around the ass cheeks region". Simply put, the hotel industry is dropping the ball in many areas where they can thrive. I would LOVE to work for a chic independent hotel doing marketing, I think I know enough about the industry and online marketing to help them immensely.

I am going to discuss it with the owner of the establishment I work at to check out the site and see what they think. They are fairly old school in their thinking and I am scared that they might fall behind. I have been trying hard to be persuasive in transitioning them into online marketing. They might be willing to test it out at their smaller property first and see how it goes. I may get back in touch with you via email in the next couple days.

All I can tell you when negotiating with sales/gm/owner is put emphasis on their value. They do want to fill a room, as a room is perishable just like a food item. A room not sold is a room that can never be sold again. Just a quick question with regards to bidding on a guest - does the hotelier see other hoteliers bidding? Or is this private?

Regardless, I am very interested to see how your venture plays out. Again, good luck!

<comment><AUTHOR> Thank you Matt :)

Yes, lodgings see the bids from other lodgings in real time, including all the details. Why?

</comment></comment></comment></comment><comment><AUTHOR> LetMeGo is a new service that allows travelers to submit their itineraries so that lodgings bid for their stays: <http://letmego.com>

LetMeGo is the result of the famous/infamous immersion discussed in here "Results of: 7 developers, working 24/7 for 90 days, 1 house (in Colombia)" <http://news.ycombinator.com/item?id=730031>

I know I don't participate a lot in HN. In part because I discovered you only a few months ago and in part because I've quite busy getting LetMeGo ready for launch. Anyway, I look forward for your feedback and to participate more and more in here.

Thanks!

</comment><comment><USER4> Not too impressed by being told Chrome isn't good enough for your site!

<comment><AUTHOR> I guess you used Chrome for Mac, right? Chrome for Windows is working fine. Maybe Chrome for Mac, too, but we haven't completed QA there yet. We will complete it soon, though.

<comment><USER5> I get the warning with Chrome 184.108.40.206 on Windows.

</comment></comment></comment><comment><USER6> I absolutely love the intro videos, especially the one narrated by Obama and Ozzy Osbourne

<comment><USER7> I agree, but I couldn't help thinking that a Christopher Walken video would really have sealed the deal for me.

</comment></comment><comment><USER8> I was confused by the video tour. I clicked watch video tour and was given the option of who to narrate it. Unfortunately I didn't know what that was. I thought I was looking at a rhetorical question along the lines of "you pick your favorite music, so how do you pick your favorite lodging?" or something like that. I'd suggest changing the heading of that to "Which narrator do you want?" or something with the word narrator.

That was my only nitpick. I tried out the interface and I love it. This is a really cool app that I find to be really well put together.. It does have the Catch 22 of you need lodgings to get users and users to get lodgings. Hopefully you find a way around that.

<comment><AUTHOR> We will improve the interface for the video tours using the feedback you and the others (below) have provided. Thank you!

</comment></comment><comment><USER9> I'd get rid of the yellow note at the bottom---at least from the home page. You could handle uncaught exceptions and then display the note, for example, or put it in the About Us page.

Us Colombians are too polite, so it would probably be OK to display upfront if it was local. The Beta logo should be enough for most people. The note just predisposes users to think something might go wrong.

<comment><AUTHOR> Indeed. The objective is to under-promise and over-deliver ;)

</comment></comment><comment><USER10> Looks good...but from a usability standpoint, would it kill you to label the fields? Which one is start date, which one is end date. What about i18n? "Where are you going?" is a little too ambiguous I think. I presume that's what you were going for, but locality can be quite granular. Hopefully there is an intuitive interpreter parsing that text.

<comment><AUTHOR> Hmmmm... Good point. In fact, there in interpreter for that, but you've got a good point. We will try to make it easier to understand. Thanks Drew!

</comment></comment><comment><USER11> The layout reminds me of the food pyramid -> [http://www.fda.gov/ucm/groups/fdagov- public/documents/image/...](http://www.fda.gov/ucm/groups/fdagov- public/documents/image/ucm070186.gif) (Not a bad thing!) except the style of the layers is not consistent. Applying some color theory would certainly help. There seems to be ~9 different fonts on the front page. As for the concept of the site: the notion of inverting the business model is really neat, I hope it succeeds.

<comment><AUTHOR> We will definitely reduce the number of fonts. And I though we had the number of fonts under control! I forgot to check the home page, though :S Thanks for noticing it!

</comment></comment><comment><USER12> I really liked in the video that you could skip ahead to different parts. What did you use to do that?

I liked the videos, and thought the different voices was an interesting gimmick, though I personally would load a video as soon as the user clicked the first link, and while it is loading give them the option to switch between voices. How many people click the first link without clicking to watch the video?

<comment><AUTHOR> Thank you! It is a feature of FlowPlayer.

We will improve the home page so that one of the videos plays back automatically. Thanks for the idea.

<comment><USER12> Cool, but don't just take my word for it set up an A/B test, there are many great ideas friends have suggested, that the internet at large was not too thrilled about.

</comment></comment></comment><comment><USER13> Looks really good, great design. A couple minor specific notes...

-"Lodgings" is an awkward word to use in the main description. I agree that it is the most accurate word to use, but I think few people think of the term "lodgings" when looking for hotels.

-Your "How it works" section looks cool, but it's an image. No text. That's not very semantic. It also means I can't copy and paste the description to tell a friend.

<comment><AUTHOR> Thank you Steve. You are right. We have had several hours of discussions in reference to the proper word to use ("lodging" or any other) and we have not been able to come up with a definite solution.

We will change the "How It Works" area today as per your suggestion. Thanks!

<comment><USER13> My vote would go for something like "accommodations". I think it's a little more mainstream.

</comment></comment></comment><comment><USER14> I love this idea. I'll try it out on my next trip. I love the videos they're awesome - although since your site looks AJAX-ie you might want to consider opening the videos in a "lightbox" popup - I tried several times to find the close button on the first video I opened - because I didn't realize it open a new page ...or you can just sum it up to my stupidity :p

<comment><AUTHOR> Thank you Tyohn. We decided to go with separate URLs for better SEO, but we will try to make it clearer for visitors :)

</comment></comment><comment><USER15> my thoughts:

too many options for the video tour. i mean, i didn't have a problem figuring it out, and i enjoyed what i saw, but you're going to find that some users will get confused. from the standpoint of converting customers, provide a default and start it playing, and then provide other options if they want to get silly.

i'm not sure how i feel about the red box on the map. i kind of think that it should be a bit more static, so that it doesn't automatically re-search each time the zoom changes or i scroll the map. but i can't think of a good solution off the top of my head.

there are a few references to st.hal.biz, where its clear that it should be just hal.biz. example: <http://st.hal.bz/img/global/guarantee120.gif> is showing up as a broken image vs <http://hal.bz/img/global/guarantee120.gif>

<comment><USER6> I agree. I would suggest having one of them already opened, with tabs for the rest.

<comment><AUTHOR> Great ideas for the videos! Thank you. We will try to implement them ASAP.

</comment></comment></comment><comment><USER16> Looks pretty cool.

When I was on the 'iternerary' view the bar at the top of the screen changing colors drove me crazy though.

<comment><AUTHOR> Thank you! Would you mind sharing the link with me? I am not sure to which interface you were referring to.

<comment><USER16> [http://letmego.com/submit/toronto?G_type=AREA&check_in=2...](http://letmego.com/submit/toronto?G_type=AREA&check_in=2010-02-12%2003%3A00%20PM&check_out=2010-02-15%2011%3A00%20AM)

Just change any of the options on the side and you'll see what I mean.

<comment><AUTHOR> I see. You are right. It may become annoying. Thank you for the tip. We will work on it!

</comment></comment></comment></comment><comment><USER17> I`m using Chrome/Ubuntu 9.10 and got the warning message. Everything worked fine though.

After selecting my travel dates, I got the impression that the whole page would scroll down, not just the left column. It was a bit confusing I think.

I´m not a fan of nesting scroll bars.

I will test the service out for real in a couple of months.

</comment><comment><USER18> I won't repeat what others said above, but I will add that I feel the home page is too gray / dark. The "search" page has a better color balance than the homepage. The way it is now it's rather depressing, and less inviting. Pretty nice otherwise.

</comment><comment><USER19> Are you able to share what strategies you have for increasing participation among lodgings businesses? It seems to me that this would be a big challenge and a huge factor in your success, and I am curious how you address it.

<comment><AUTHOR> <http://news.ycombinator.com/item?id=1066008>

</comment></comment><comment><USER20> great idea and great execution (from what I can see). The narration idea is very original with the video intro. But I do agree with other commenter that it was odd to have to pick a narration. I'd suggest you pick the one you like best or which best shows off the capabilities, and just use that, or rotate through them. Even though you went through the effort to create the different videos, it is best to keep something like that simple for the user.

</comment><comment><USER21> Where did you get all of the hotel data (rating, amenities, types, of rooms, pictures, etc.)? Are the hotels that sign up required to submit that information?

<comment><AUTHOR> Yes. They are required to sign up and provide all the info.

</comment></comment><comment><USER22> Cool looking site .. but where did you get the voice from?

<comment><AUTHOR> My other startup: <http://voice123.com>

</comment></comment><comment><USER23> Great idea. I will use it next time I travel and if I get a great deal I'll be a loyal customer.

</comment><comment><USER24> Damn. I was hoping you were making an app to let landlords bid on desirable tenants.</comment>


Zynga Chief Seeks to Claw Back Stock - <AUTHOR> http://online.wsj.com/article/SB10001424052970204621904577018373223480802.html?mod=WSJ_hp_LEFTWhatsNewsCollection <comment><USER4> Anytime you have both vesting schedules and at-will employment, your employer can fire you at any time and you will not get any unvested stock. Startups need to do this all the time when employees are underperforming or a bad fit. Renegotiating so that you can get a portion of that stock and stay employed is perhaps, if this were an isolated incident, a much better deal for the employee than getting fired.

Where this becomes very distasteful is if these aren't isolated incidents of very underperforming employees but Zynga using its leverage to negotiate compensation down arbitrarily. Even if a developer is performing just as well as expected upon hiring, it is unlikely that they could get a guaranteed offer of $1M from another company, and so Zynga could easily force a renegotiation of an unvested grant worth $5M down to $1M. This would be reprehensible - <USER2> if that employee isn't really adding $5M worth of value, if they are performing up to the expectations that were set when they were hired then the agreement should be honored - getting a chance of a huge upside is one of the reasons employees take lower salaries and work longer hours at startups in the first place. A company that abused its bargaining position like this should not expect to be able to hire good employees in the future.

There isn't really enough information in the article to know that this latter case is what's happening. There are scenarios in which this behavior is very malicious, and scenarios in which it's relatively reasonable. It all hinges on how the employees in question were performing and how common this tactic is. I don't think anybody here knows those details, so we should really try and avoid the typical internet rush-to-judgement here. I don't know anything about the internals of Zynga, but I have seen HN get out the pitchforks for other companies when the real story turned out to be much more mundane.

<comment><USER5> The oddball thing here is not that people are being let go so that a startup can recapture their unvested stock (as you note, this happens all the time, for reasons both good and bad) but that a company would create what amounts to a "hit list" in order to systematically pressure employees to surrender shares on threat of being fired. The company-wide message to employees is, in effect, "we lured you to join us with promises of a high upside via this stock grant in case we succeeded but, now that this has come to pass, we think you are getting a windfall and want a good part of it back." Since employment is at will, this tactic can be effective in reducing the equity base and maximizing value for other shareholders in the company. But at what price? This effort is being driven by a CEO who is clearly dominant, holding almost 40% of the voting power in the company and using it, for example, to cause the company to buy back nearly $110M worth of his personal stock pre-IPO. The overall effect, then, is one of a leader who appears arbitrary and unfair to those who were there early on. While cashing in for himself, he is "clawing back" on others. It might be legal but it just plain looks bad and would hardly seem to be a good way to motivate people or to keep strong people in your company for the long term.

<comment><USER6> No one should be surprised that this is coming from Mark "I Did Every Horrible Thing In The Book Just To Get Revenues" Pincus.

It not just looks bad. It IS bad. Arbitrarily deciding who does and who does not deserve to vest is ridiculous. If the person is not performing, let them go. They are an at-will employee.

By negotiating them to keep their job by giving stock options back, they are simply trying to keep that employee at a cheaper rate. The employees must be performing effectively if they want to keep them. The notion that these people are doing nothing and still employed by Zynga is disingenuous.

The naivete (or CEO apologism) around here is astounding. I understand keeping an unbalanced and impassionate point of view but at some point if it sounds like a duck and looks like a duck it IS a duck.

</comment></comment></comment><comment><USER7> I'm going to take a slightly controversial position here - so please read through my logic before you downvote me into oblivion.

If you read the article, you'll see that what's happening at Zynga is not "Taking Back" stock, instead it's talking about _future_ compensation. Every time I've been through a Compensation Review - one item that is made very clear to my manager, is how much _unvested_ stock I have in the company. That plays a role when they reviewing my salary and bonus, because Unvested Stock is a component of my forward-looking annual salary.

All Pincus is doing - is looking at his executives, and saying, "Hey, You are going to make $10 Million dollars NEXT YEAR, but you aren't doing $10 Million dollars worth of work - we're going to have to bring your NEXT YEAR's salary down to $1 Million" - He isn't talking about stock already owned by the employee.

California is an At-Will employment state, Zynga is located in California - Pincus could just fire these people and be done with it - as long as he wasn't discriminating unfairly (Based on Age, Race, Ethnicity, Marital status, or any of the other protected classes) - it would be completely legal, and, if these people weren't performing, completely rational, albeit really quite brutal.

In the scheme of "How can my company screw me on stock" - I put this in the 50 percentile category.

With all that said - dick move. I wouldn't want to work for someone who would do this to me.

<comment><USER8> I have to respectfully disagree with your logic. If you read the article, what is actually happening is they are retroactively trying to reduce restricted stock grants _that were already made_.

So, if I say "come work for me for free, and in exchange I will give you 10,000 RSU that vest over 4 years", then 3 years down the line say "just kidding, you don't get these last 2,500, but you get to keep working here", that is absolutely trying to take back comp. They can certainly withhold _future_ restricted stock grants, but trying to take back already granted shares, vested or not, is beyond the pale in my opinion.

<comment><USER7> They can't touch the shares that have vested - I think we both agree here. They could just fire the non-performing executive, and put an end to the vesting schedule.

Here is the thing - Firing Non-Performing executives at startups with huge stock allocations happens ALL the time at _every_ company. It's not at all unusual - I've never been at a company where it hasn't happened. And frequently. It's absolutely business as usual.

I don't understand why Pincus didn't just fire these people and move on. That's the weird part of this entire thread.

<comment><USER9> Good point. I'm positive I'd just fire them. It sounds like he made a list of B/C players but wants to keep them around.

Maybe he's concerned how it would look if he fired a group of execs right before the IPO?

</comment></comment></comment></comment><comment><USER10> Wow this should be straight up illegal and yet:

"One lawyer said that over the past year, he has heard executives of three social-media sites discuss the possibility of clawing back equity from some employees. Another lawyer, who has handled stock-compensation issues with technology companies for decades, said he never saw a company try to take equity from employees until about two years ago, but has since seen three such cases at start-ups."

I don't see how this is anything other than theft. They're saying that either you give us some of your compensation back (you may not have joined if it were not for those stock options/grants) or you're fired and lose it all. Insane.

<comment><USER11> They're not saying "you give us some of your compensation back", they're saying "your future compensation will be less than you expected."

It's kinda like if someone gets you to join their company by offering to pay you 100k/year, but then 6 months in drops your pay to 50k/yr.

And really it's not even that. It's "you got hired at 100k/yr, but then the company was WILDLY SUCCESSFUL and you then expected to make 10M over the next year but they're dropping your pay to 1M for that year"

Skeezy? Definitely to some degree. Theft/Illegal? I don't think so.

<comment><USER10> The difference between a pay cut and a stock restriction is that pay is regular and vesting is not.

Let's say you renegotiate down 50% like you suggested. That means from now on you make 50% and before you made 100%.

A vesting schedule though almost always accelerates towards the end. So if you renegotiate down 50% towards the end you may in fact be likely losing 75% or more. That's the problem with renegotiating stock grants/options, you likely still have a large amount left unvested and you put in all the hours and work beforehand with the promise that loyalty would be rewarded over time.

<comment><USER11> > A vesting schedule though almost always accelerates towards the end.

Really? That's hasn't been my experience. In every situation I've seen (or even heard of) vesting has been on a regular schedule with the exception of a cliff at the beginning of employment (generally 1 year).

If the zynga situation involves people who have not yet reached their 1 year cliff the skeeziness factor would go WAY UP imho.

</comment></comment></comment></comment><comment><USER12> I think it is really important that people understand that stuff like this can happen even when you _win_ the 100 to 1 bet that you are working for the next Zynga and not the next $NAME_A_STARTUP_RANDOMLY.

<comment><USER13> Another potential gotcha is the post-IPO sales lockout period. I don't know if it's the same in software, but in a previous life I was a chemist in a biotech startup that had an initially successful IPO. The rank and file were prohibited from cashing out options for six months after the IPO but somehow all the execs were exempt and got rich while the stock was enjoying a post-IPO high. Needless to say, by the time we peons could sell the value had plummeted.

<comment><USER14> Thats what put options are for.

<comment><USER15> Puts don't start trading immediately after the IPO. (There are also lockout/reporting requirements on derivatives, but I've always wondered how enforceable/detectable such transactions are.)

<comment><USER9> There are ways to hedge though, right?

<comment><USER15> Not usually, you need someone on the other side of the transaction. And even if the options were available, the volatility would make the spread very costly.

<comment><USER16> The standard way, if you are prohibited from trading that stock, is to buy derivatives that are short on a basket of similar stocks. This hedges against a general market fall, such as 2001.

</comment></comment></comment></comment></comment></comment></comment><comment><USER17> Silicon Valley isn't a special place for invention and for startups because of special infrastructure or special laws, but because of special ideas and special norms. Those norms don't hold up in court, and they erode a little bit every time someone like Pincus only thinks about what they can get away with instead of what's right. This reduces the perceived value of equity for everyone in the valley, makes everyone less open, less trusting, less willing to take a risk. In the end, we all lose for that.

If I had my way, Pincus would never work in this town again. Anyone involved in this decision should be made a pariah. If we don't defend the norms that make innovation possible, we'll lose them.

</comment><comment><USER18> Grellas, PG, or others with experience in this field -- is there a legal document that I can make, as a startup CEO, that prevents my company from doing this in the future?

I would never do something like this but I want that to be legally "handcuffed" so that no employee can ever think that we would/could do this.

<comment><USER19> It seems that Zynga is using a provision that says that all unvested stock grants are returned if the employee is fired. This is a pretty common provision from the contracts I have seen.

They then seem to be saying "give us back some of the stock voluntarily or we will fire you and you will lose it all."

If you want to make this impossible you can make your contracts such that unvested stock is not lost when one is fired. However, in that case you may end up with a lot of stockholders that are previously fired employees which may not be good for the company.

You can put in a provision in the contract that says that people can only be fired for good cause and that cause cannot be that they did not want to give their stock back. However, while this seems a very fair provision, it will invite a lot of lawsuits and may cost you a lot of money even if you do follow it.

You can put in a provision that says that one cannot ask employees to give back shares in exchange of not getting fired, but it is doubtful that will be effective. The courts usually allow contracts to be renegotiated if both parties agree to it, even if the provision that is to be renegotiated is the one that says "no renegotiation."

So, honestly, I cannot see a good way to "handcuff yourself" as you put it. You may as well just rely on being a decent person.

__________

PS: None of this is legal advice. This is just hypothetical discussion regarding a hypothetical situation. Please do consult a lawyer for your specific situation.

<comment><USER20> Employees can't rely on your decency because the investors might put someone else's finger on the trigger if that time comes. I wish there were a way to ensure everyone gets paid or screwed together, but tactics like share classes and dilution and liquidity preferences seem to be so strongly customary that I wonder if a VC would lose interest in a firm that relinquished the power to selectively starve that VC's rivals at the banquet.

</comment></comment></comment><comment><USER21> So their CEO has a list of executives that he feels did so little he considers them "missing in action". If that were the case, why didn't he just fire them for cause? That would have stopped the vesting clocks on their options.

Or was he paying so little attention to who was and was not contributing that firing them didn't occur to him at the time? In that case, who's really "missing in action" here?

If I were going to take my company public soon, the impending litigation shitstorm is exactly the kind of thing I would not want to have hanging over me.

<comment><USER15> It communicates to IPO investors: CEO is such a dick that he will happily screw over employees in order to enrich himself.

Believe it or not, there will be a coterie of investors who will get behind that. (They will perceive that their interests are aligned with the CEO after IPO.)

<comment><USER22> As an investor what this tells me is the CEO has no personal integrity, and I begin to doubt the veracity of his IPO filings.

<comment><USER15> You shouldn't believe them anyway, because they are selling you something.

<comment><USER22> You're probably right about that. Still, I prefer not to do business with people lacking integrity. There are a lot of different ways to get swindled.

</comment></comment></comment></comment></comment><comment><USER23> This is bad for the startup community. There isn't much reason for a great hacker to join startup if he/she thinks you might want to play 'take-backs' if the company succeeds.

<comment><USER7> Well - the hacker can be fired. That also takes back all the unvested stock.

</comment></comment><comment><USER24> What's really disgusting in the conversation is that there's some implicit assumption that it was _wrong_ for the Google Chef to have received $20mm. Indeed, the only thing wrong with that is that $20mm is far too little, given the Chef's contributions to Google (I am assuming they're talking about Charlie?)

</comment><comment><USER20> > The equity recouped has been valuable as Zynga recruits experienced > executives.

The obvious question is how they could ever expect to convince executives they won't be defrauded the same way a couple of years from now. This tactic seems so unscrupulous as to be self-defeating.

</comment><comment><USER25> On the one hand, it's kind of a classic dick move by Mark, pretty typical of his bloody mindedness. (He's more of a bulldog than Zinga ever was.)

On the other, in any company as big as Zynga there are people who coast along and don't really earn their shares. Zynga isn't trying to take back what's vested, just clamping down on future shares for people who aren't doing a great job, maybe playing a lesser role than they were originally hired for. That seems like a perfectly valid thing to do -- in fact it's only fair to the employees who are pulling their weight.

<comment><USER26> This is ludicrous. If people are coasting in a company, those people should not stick around (dis-incentivized is a word I've heard used before).

The fact that this is happening right before launch is pretty reminiscent of Skype's fiasco prior to exit (sale to MSFT). In fact, both of these companies have one VC that's in common, Silver Lake Partners: [http://www.silverlake.com/partners/investments.php?page=inve...](http://www.silverlake.com/partners/investments.php?page=investments&id=43)

</comment></comment><comment><USER27> Can someone summarize the rest of this post for those without a subscription to WSJ?

<comment><USER28> Copy and paste the URL into Google. Then click the link when it shows up in the SERPs. It should get you past the paywall.

<comment><USER27> Thanks!

</comment></comment></comment><comment><USER29> More often, startups fire employees under the disguise of performance. I am not sure what constitutes non-performance but it has been used countless times as an excuse for getting back the unvested stock options.

</comment><comment><USER30> I guess they want to make sure people don't want to come work for them...

</comment><comment><USER31> _But several lawyers say the practice has been so rare it hasn't been tested in court._

Hopefully it remains rare. What a dick move. Bad for startups everywhere.

</comment><comment><USER32> A rich man who made his money any way he could, ethics be damned, is trying to steal money from the people who helped make him incredibly wealthy. Is this really shocking?

<comment><USER33> "It made him sick to have to turn money over to the guys who stole it. He'd rather whack 'em." -Goodfellas

</comment></comment><comment><USER34> This sounds like a good example of why 'at-will' employment should be abolished and why there should be more employee rights.

There is a power inbalance here. Companies give stock, but that can be taken away if the company fires you. Even if you work hard and are a good worker, the dollar price of what they get if they fire you might be too high. The best way to stop this is rebalance the power. Prevent the company from being able to fire you for no reason.

</comment><comment><USER35> How much of Pincus' stock is getting clawed back? If this really is a problem, isn't it afterall, his fault in the first place? So shouldn't he suffer?

<comment><USER36> He's an enormous holder - the article cites almost 20%, and almost double that in voting rights. It's hard to believe his contribution is worth $4B (1/2 of what Steve Job's net worth was) if the $20B valuation holds up in the IPO. Disgraceful.

<comment><USER37> Why? Zynga hit the (hypothetical) $20B in far less time and employees than Apple did. That is far more return on invested capital.

</comment></comment></comment><comment><USER38> FWIW, you cannot get any Zynga employee's opinion in this because the pre-IPO "quiet period" is in effect. So this is a pretty one-sided conversation.

</comment><comment><USER7> None of these employees who have had their stock (possibly millions of dollars of compensation) "clawed back" will be likely to do any productive work whatsoever for Zynga - so why not fire them and get it over with? After all - that would have the effect of instantly canceling all future vesting - <USER3> discussion or agreement with the employee required whatsoever, and zero controversy.

After some deeper thought on this topic - my new assessment is that these people are actually experiencing what is called a "Soft Termination" - they've been fired, but they are allowed to stay on as employees for things like Medical, Job Hunting, etc....

Instead of appreciating it, though - they are squawking about having their stock pulled back in return for not being "officially terminated". I bet this type of thing goes on more often than we know, particularly with Executives who may have a significant percentage of the company and are not working out.

<comment><USER37> Bingo. If a company stops paying you as much, stop working as much. If you are annoyed that you aren't getting paid for as much as you were contributing before, you learn for next time to pay attention and adjust course before waiting a whole year for a payday.

</comment></comment><comment><USER39> So why do employees accept stock grants without a clause stating "If you choose to fire me without cause, (x% of) my unvested shares shall vest immediately"?

</comment><comment><USER40> There's the obvious J. P. Morgan quote:

Asked: "Is not commercial credit based primarily upon money or property?" "No sir," replied Morgan. "The first thing is character." "Before money or property?" "Before money or anything else. Money cannot buy it...Because a man I do not trust could not get money from me on all the bonds in Christendom."

</comment><comment><USER37> The expected value of illiquid non-controlling stock is 0. There is no problem here. Don't take a job that pays you in unicorn horns and pegassus tails, folks.

<comment><USER41> The term "expected value" has a technical meaning, and is not zero here.

Your point that people should not expect to make money from illiquid non- controlling stock is reasonable, though.

</comment></comment><comment><USER42> It's sad to say, because I hate unions, but in this case Unions would solve this problem.

Evil likes this, creates a need to create a counter force.

</comment><comment><USER43> The real news is not that they're taking back stock, it's that their initial stock plan was so screwed up that they have to do it.

<comment><USER44> I thought that myself. "Wait, shouldn't they _know_ they've been giving out more stock then they should have??!"

</comment></comment><comment><USER45> Mark Pincus, the scum king of lead gen [1:5], is behaving in an unethical manner? I'm shocked. Shocked!

Is it rude to think these employees laid down with dogs and are bitching about fleas?

[1] [http://techcrunch.com/2009/10/31/scamville-the-social- gaming...](http://techcrunch.com/2009/10/31/scamville-the-social-gaming- ecosystem-of-hell/)

[2] [http://techcrunch.com/2009/11/07/horrible-things-slink- back-...](http://techcrunch.com/2009/11/07/horrible-things-slink-back-into- zynga/)

[3] <http://cdixon.org/2009/09/28/the-new-economy/>

[4] [http://techcrunch.com/2009/11/08/zynga-to-stop-all-in- game-o...](http://techcrunch.com/2009/11/08/zynga-to-stop-all-in-game-offers/)

[5] [http://techcrunch.com/2009/11/01/scamville-hotornot- plentyof...](http://techcrunch.com/2009/11/01/scamville-hotornot-plentyoffish- facebook-myspace/)

<comment><USER46> I'm surprised that anyone is surprised. It's _Zynga_. Their business model has always been slimey, and they add no value to society. Net negative, actually.

<comment><USER47> Not only do they add no value to society, one could argue they are responsible for decreased productivity of employees who waste their work day playing Zynga games or come back to work tired after wasting 2 hours at night planting cucumbers on FarmVille.

<comment><USER48> Can you help me understand the contempt you have for the Zynga games? I don't play, myself, but I do understand that the games exploit (negative? compulsive?) psychology, and they've used unethical business strategies.

But who cares? After all, it's unlikely the next Vonnegut is going to spend his time playing MafiaWars instead of writing "Slaughterhouse Five" or anything.

I am reminded of the aphorism, "Time you've enjoyed wasting is not wasted time."

<comment><USER49> Do you know any writers? Because the ones I know mostly have a terrible problem with procrastination, avoidance, etc.

Fulfilling a compulsion is not actually enjoyable. So unlike most game-makers, I don't see them as creating fun.

<comment><USER50> I've never really played any of these games, do they not involve at least some level of skill or strategy?

What makes them worse than say, world of warcraft?

</comment></comment></comment></comment></comment></comment><comment><USER51> I've seen people get cheated out of compensation this way, in some cases where they took greatly reduced salaries, in exchange for shares, but the shares vested, and after creating the major innovation the company wanted, they were fired without cause before the first vesting cliff.

I think vesting, as a mechanism, is problematic. Especially when you make a founder vest stock they've already bought and paid for (with sweat equity and the money they put up to start the business.)

Unfortunately, a lot of startups play fast and loose with this. Its common to ask someone to take a lower salary, where, say, they're making $5,000 less each month, but they aren't vesting any shares each month. Some companies only vest in blocks each year. Which means that if employment ends for any reason, the employee will effectively be shortchanged for the months of work before the cliff.

Yet it is pitched as if the stock or options is like the salary.

Its not uncommon for startups to fire employees right before their first cliff... if there is a shorter downturn in outlook (or if the founders are on the down side of the roller coaster and are panicking...)

I think a better solution would be for any kind of equity reward to be given monthly, with the paycheck.

Rather than grant a huge block of shares, vesting over 4-6 years, maybe give the employees a small amount each month. This amount would be fully vested, and it can be adjusted whenever you have your reviews.

As the startup grows, naturally, the shares will become more valuable, and the grants in future years will likely be lower for everybody. They could just decline faster for the under-performing employees and maybe not even decline at all for the star performers.

This eliminates the stress and unfairness of cliff vesting.

For the situation where an employee might leave within the first year, you could have a clause that gives the company the right, upon termination, to buy back the employees equity at the highest price during the year (e.g.: if you raised two angel rounds, then it would be at the higher) plus a dollar a share. This way the employee isn't screwed over, and you don't have people who were there for just a year clogging up your investor rolls. If it isn't buying back the stock at the price you've been selling to investors (plus a dollar a share) then your expectations of the value of that stock are so low that its wrong to cheat the employee out of the roll of the dice for a big upside.

I might be missing something, but this is an attempt to create something fair, for an environment where cash is short and people want to use stock or options as regular compensation.

Paychecks don't vest, and imagine how many people would work for you if you said "We'll pay you $100,000 a year, but only on January 1st of each year. If you leave before then, or we decide to fire you, you get nothing."

<comment><USER52> Viewed from an employers' point of view, the cliff means you've got to put in some real sweat into it before the shares become worth anything. How long do you think it takes for a new employee to be truly adding value? Perhaps it's not a year, but I suspect many eat a lot of time and slow things down for a while until they get comfortable. I think the cliff is just a balancing of those interests.

The last round+dollar per share seems silly. First because shares are somewhat arbitrary, it's the % of the company that matters. Also if there is a down round, why should the employee benefit? The whole company is struggling, why should there be a bonus for leaving/getting fired?

It seems to me that if you want to work for a startup and are taking equity compensation you should understand the RISK involved. You should be prepared to accept the pay and be comfortable knowing that your equity, real or imagined, could be worth nothing.

That said, the cliff issue could probably be negotiated more favorably for employees and screwing over your employees seems like a horrible policy in a world where every tiny detail of your life could possibly be published and forever archived publicly.

<comment><USER53> Founders only need to exit once. There is surprisingly little incentive for a founder to avoid screwing over employees.

<comment><USER38> Spoken like a true non-founder... Reputation is everything.

If a founder ever wants to start a company again, screwing over employees will haunt him in the long term much worse than any legal repercussion that can be fixed with money.

So I wager there is a lot preventing founders from screwing employees, specially in startups, where people are hyper connected.

The comments in this story are running rampant with hearsay and people who have no involvement making up facts and jumping to uninformed conclusions. Zynga employees cannot comment on anything anywhere because of the quiet period.

<comment><USER54> > Reputation is everything.

Not if you, as founder, really are in it for the money and your first exit sets you up for life. Can anyone think of a company that might be relevant to this discussion where such a description might apply?

> Zynga employees cannot comment on anything anywhere because of the quiet > period.

I can't believe no-one is going to sue over this, given the amounts of money presumably involved, and I doubt a judge is going to seal the records in such a case. I'm sure we'll find out the details soon enough.</comment></comment></comment></comment></comment>