title: Is it possible to use a broadcast inside a Spark partitioner?
body: <p>Suppose I have a custom Spark partitioner that, to operate, requires a fixed, very large collection, passed in this example to its constructor. The <code>getPartition</code> method uses this collection to return the partition number.</p>
  
  <p>I'm thinking to wrap the collection in a <code>Broadcast</code> variable, and pass the <code>Broadcast</code> to my custom partitioner constructor. Can a <code>Broadcast</code> be used outside of the typical functions passed to RDD methods like <code>map</code> and <code>filter</code>? Is its behavior well defined in this case? Is it guaranteed that the variable is transferred and shared efficiently among worker nodes JVMs?</p>
  
  <p>Does it make any sense or it doesn't improve anything compared to just letting Spark capture the collection the standard way?</p>