How can I set the index fill factor back to server default?
Through testing, it seems as though to get back to the default fillfactor on an index you need to drop and recreate it.  When you do an ALTER INDEX ... REBUILD, any unspecified parameters will remain the same in the index's metadata unless otherwise explicitly changed in the ALTER INDEX command.

drop index IX_YourIndex
on YourTable
go

create index IX_YourIndex
on YourTable(YourKeyColumn)
go


To verify this:

select
    name,
    fill_factor
from testdb.sys.indexes
where name = 'IX_YourIndex'


fill_factor should be set to 0 to denote that it is using the instance's default value.  (Note: 0 is for the default value, even if the instance configured fill factor is set to 95)