 Aside from public life, de Silva's central passion was stamp collecting.His Ceylonese collection is said to have been world-class, second only to that of King George V. He was said to be one of the most notable philatelists in the world and also owned the legendary orange-red "Post Office" Mauritius One Penny (1847) stamp; considered to be among the rarest and most expensive stamps in the world.In keeping with his charitable ways, he donated the stamp to his relative, Sir Cyril de Zoysa, for the construction of the YMBA headquarters building.