title: How can we build multi tenancy on top of hadoop ecosystem?
body: <p>We are trying to build multi tenancy on top of hadoop ecosystem.</p>
  <p>our ecosystem typically comprise of hadoop components like hdfs, yarn, hive, oozie, zookeeper.</p>
  <p>Till now, I have looked onto concepts like</p>
  <ol>
  <li><a href="https://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/Federation.html" rel="nofollow noreferrer">HDFS Federation</a></li>
  </ol>
  <blockquote>
  <p>It federates your distributed storage(HDFS) with the help of a seperate namenode for each federated HDFS partition.</p>
  <p>Problem: Say you have 2 tenants for a single cluster hence 2 namenodes, 2 namenodes will imply 2 hive servers, 2 oozie servers and similarly 2 of each of other hadoop components which will communicate with their respective namenode and write in their respective HDFS partition.</p>
  </blockquote>
  <ol start="2">
  <li><a href="http://hortonworks.com/blog/multi-tenancy-in-hdp-2-0-capacity-scheduler-and-yarn/" rel="nofollow noreferrer">Capacity scheduler</a></li>
  </ol>
  <blockquote>
  <p>compute quotas could be enforced for each tenant.(say tenant 1 - 50% and tenant 2 - 50%)</p>
  </blockquote>
  <ol start="3">
  <li><a href="https://www.mapr.com/blog/get-real-hadoop-true-multi-tenancy#.VZ9qvl-qqko" rel="nofollow noreferrer">MAPR multi tenancy features</a></li>
  </ol>
  <blockquote>
  <p>I could not find how we can develop a similar multi tenant environment for hadoop cluster.</p>
  </blockquote>
  <p>What I was thinking to do was like each tenant(with its users) will have a</p>
  <ol>
  <li>compute quota(through capacity scheduler)</li>
  <li>HDFS directory for each tenant in HDFS (like /usr/tenant1, /usr/tenant2)</li>
  </ol>
  <blockquote>
  <p>All the users belonging to tenant1 group will have access to write in hdfs directory(/usr/tenant1/username)</p>
  </blockquote>
  <p>What problem I faced was..
  say i create a table from hive as a tenant1 user, it created the table /apps/hive/warehouse similarly /apps/oozie/data. what I was thinking was it will create table in user's hdfs home directory and hence only tenant1 users will have access to it. but it didn't happen.</p>
  <p>Can I get some help in how should I proceed with my development to create multi tenancy over Hadoop ecosystem?</p>