Fisher was born in San Francisco, California, and spent his childhood in the then-middle-class Sea Cliff neighborhood of San Francisco.
Fisher was born in San Francisco, California to Sydney Fisher, businessman, and Aileen Fisher, a cabinetmaker. He spent his childhood in the then-middle-class Sea Cliff neighborhood of San Francisco.