Skousen was born in 1947 in San Diego, California, and grew up in Portland, Oregon.
Conservative political commentator and survival strategist Joel Skousen and linguist Royal Skousen are his older brothers, and W. Cleon Skousen, the late conservative author, was his uncle.