all during the building of marble house , which took place behind high walls in closely guarded secrecy , wild rumors swirled around the servants ' halls : of a ballroom carved from gold ; of an entire medieval museum inside for the old man 's pleasure ; of entry doors weighing as much as a coach-and-four .