Performance first low, then high after adding index, then keep high after drop index
Most likely the statistics on the field were out of date. Adding the index created/updated statistics with a full table scan. Or creating the index pulled all the data you need into memory, like any scan would.

Adding an index will only perform better if it is being used. Add the index and then check the query plan. If the stats are up to date and the query optimizer elects to use it there is a big chance it will improve performance. Removing statistics has no use as it will result most likely in a suboptimal query plan.

More info on statistics https://mariadb.com/kb/en/mariadb/documentation/optimization-and-tuning/engine-independent-table-statistics/