INI configuration handler
I recently got told that LINQ slows down your code. 


Tell them they are wrong.


  Is there any better way to do this?


No. You first code for convenience and pretty code.


  Is the speed much different?


It doesn't matter unless you need to optimize it for speed but you do this only for parts that you have and can measure. You don't optimize code that you believe might by slow because someone heard from someone who heard from someone else that something might be slow.



Your ConfigHandler doesn't have to be IDisposable. You are not using any resources and clearing a dictionary isn't doing anything useful.




if (!File.Exists(configFile) || FileUtilities.IsFileLocked(new FileInfo(configFile)))               



You should check only one condition at a time and log a precise message. 


  "Either your config file couldn't be found, or its open somewhere else."


You are not helping yourself with this message. Each time any of these occurs you'll need to check both cases instead of only one.