sql server spatial index performance
That your indexing is slowing some queries but not others has a lot to do with the way spatial indexes work:

The spatial index is broken into 4 levels which are broken into grids.  With each level you will be able to specify a density which of low, medium or high corresponding to 16, 64 or 256 cells respectively. Then within each cell these are then subdivided into the next lower level with its setting.

All of this means that depending on your settings and your data distribution you may need to adjust the density at different levels to get proper index performance.  This may also mean that you need to balance your performance of your main queries of this data against your edge cases.

Take a look at this Microsoft TechNet library article for a more detailed explanation:

Spatial Indexing Overview