Cassidy grew up in Los Angeles and New York City, attending boarding school in Bucks County, Pennsylvania, and graduating from Beverly Hills High School.
Alongside his parents, and half-brother David, Cassidy was represented by former table tennis champion Ruth Aarons, who came from a theatrical family and found her niche later in life as a talent manager.