Find times when table is least frequently used
SQL Server doesn't keep track of this information. You'd have to take snapshots from sys.dm_db_index_usage_stats frequently enough that you can graph activity, and you also need to account for service restarts, index rebuilds, etc. which can remove rows from the DMV and set the activity back to zero (this may look on your chart like no activity but it really means it has been reset).

This is not a straightforward task. I think you should add more information to your question explaining why you need this information, because maybe there are other ways to accomplish your real goal (knowing when a table is read less frequently is not a goal, it's the path you've chosen to your goal).