Cannot remove item from Finder sidebar
Ok, I'm not going to pretend I know what's happening here but try deleting the file where that info is stored:

$HOME/Library/Application Support/com.apple.sharedfilelist/com.apple.LSSharedFileList.FavoriteItems.sfl2


I would bet that there's something corrupted in there.  Delete it, log out and log back in or reboot, and try to remove it again.  See if that helps.