The genre is said to have started in 1764 with Horace Walpole 's book The Castle of Otranto . As a genre , it is generally believed to have been invented by the English author Horace Walpole , with his 1764 novel The Castle of Otranto .