A tractor pulled up outside the building, in Morledge, at 07:10 GMT and deposited the dirt, police said.
A note left inside the abandoned vehicle said the action was a "personal protest against the severely flawed judicial system".
The 49-year-old man was arrested on suspicion of criminal damage and later released on bail.
Updates on this story and more from Derbyshire
The note left in the tractor read: "What a shame to have to resort to this, today is a personal protest against the severely flawed judicial system that has consistently failed to listen to simple truth and reason.
"Sorry for any inconvenience caused (to the general public)."
BBC Radio Derby reporter Nigel Cash said: "The pile immediately behind the trailer is probably a couple of feet high and couple of feet wide.
"It looks as if whoever has dumped it there may have deliberately spread it out."
Police removed the tractor and spreader after a few hours.
They said the man was bailed while their investigation continues and they "consult with the Crown Prosecution Service".
Anyone who witnessed the incident or has any information has been asked to contact the force.
A man has been arrested and bailed after muck was spread across the steps of Derby Crown Court.