Is it imperative that I backup my index files?
The first point to consider that a clustered index is actual table data.  So in that alone, I'd say yes.  Your whole database should be backed up and able to restore (there are a few exceptions to that, like read only filegroups that are subject to no DML).

The second point is that restoring a database does not rebuild or reorganize any indexes.

So yes, you should be backing up all of your data, indexes included.