Sql FILESTREAM garbage collection issue
It seems like insufficient info to tell something definitely.  For example, are you deleting through Transact-SQL or through Win32API, etc.

The FILESTREAM data is not deleted immediately from file system because SQL Server transaction logging under full and bulk recovery models permit the crash recovery. 

Have you deleted with CHECKPOINT 

delete from tablename CHECKPOINT


or tried to execute CHECKPOINT statement or use simple recovery model?  

Also, IMO,:


changing of FILESTREAM (re)configuration require restart of SQL Server
FILESTREAM data is restricted to LOCAL only drives  
FILESTREAM operations depend on hardware, see Paul Randal's white paper  
"deleting or renaming any FILESTREAM files directly through the file system will result in database corruption"   


Check Security and Reliability sections of Paul Randal's white paper  

Related  discussions containing many subreferences:  


SQL Server Filestream - Delete “speed”
How to force the filestream garbage collector to complete its work with the highest priority?  


Update:
Have you checked permissions of FILESTREAM (Windows) share/container vs. account under which SQL Server runs  under. It should have local administrator permissions.   It is recommended that no other account be granted permissions on the data container

Update2:
From Paul Randal's white paper:   


"The file system access open operations do not wait for any locks. Instead, the open operations fail immediately if they cannot access the data because of transaction isolation. The streaming API calls fail with ERROR_SHARING_VIOLATION if the open operation cannot continue because of isolation violation"   
"Antivirus software ... access to the BLOB data in the affected file will be prevented, and to SQL Server the file will appear to have been deleted."   
"Note that for a table to have one or more FILESTREAM columns, it must also have a column of the uniqueidentifier data type that has the ROWGUIDCOL attribute. This column must not allow null values and must have either a UNIQUE or PRIMARY KEY single-column constraint"    
"anything that can prevent transaction log truncation may also prevent a FILESTREAM file being physically deleted. Some examples are:"   
FILESTREAM data containers cannot be nested


Run
DBCC CHECKDB   

Update3:
I cannot coach you online. This is Q&amp;A board. I already had been banned many times for infringing the rules on StackExchange having discussions instead of posting question or answer  

Run 

DBCC CheckDB (QPS8, repair_rebuild)

DBCC CheckDB (QPS8) WITH No_INFOMSGS, ALL_ERRORMSGS


and put output here.     

To check/set NTFS permissions: right-click on folder in Windows --> Properties --> Security  

To check/set MSSQLServer:  in command line type  

SQLServerManager10.msc 


navigate to SQL Server Services and double-click corresponding instance of SQL Server   

Update 3b:
Probably, you should drop and recreate index or you have further deeper issues and you should restore database from backup or perform database repair permitting loss of data. This is quite separate topic/question.
I do not want to take any responsibility to guide you on it.