I don't ever use Siri, but it's using too much battery
In iOS 9, Proactive services like Siri Suggestions (on the far left home screen page) will consume power. It has nothing to do with the actual Siri.

If you don't like Siri Suggestions, you can disable it in Settings > General > Spotlight Search.



You might also go into Settings > General > Siri and make sure "Hey Siri" is turned off, especially if you have an iPhone 6s or 6s Plus. Your iPhone "listens" for the words "Hey Siri" but only when connected to power (unless you have an iPhone 6s or 6s Plus, then it's all the time), but doing the two things above and then restarting your device should prevent more usage from accumulating under the "Siri" bucket.