How much space is needed to reclaim space from a table in MyISAM
As far as I can tell, the documentation is not very clear on the subject. 

From my experience, I'd say it always rebuilds fresh table files for data and index.

Intuitively, from what I know about MyISAM storage engine, I'd say it should be possible to run a full data file in place optimisation if and only if the rows have constant length. That is if you don't use columns with dynamic lengths (VARCHAR, TEXT, etc.). 

Regarding index file, rebalancing the trees might be possible in place, but at a higher cost (duration).