What breaks if I delete the folders named Public and Sites?
As an alternative to deleting them, you could consider hiding them in the Finder instead. To do this, launch Terminal then enter this command:

chflags hidden ~/Public ~/Sites


If you ever want to make them visible again, you would use this command:

chflags nohidden ~/Public ~/Sites