How to seal a unused thimble in exhaust chimney
I think you can but I also think you are going to loose this fight.  Do what the inspector says.  Life will go easier for you.  If in the future, someone wants to install a wood stove in that spot, it is not that hard to re-chink a hole. (BTW the oil furnace probably should be vented through a chimney liner.  And any future wood stove will probable be required to have its own flue-way.)