Is there a way to reduce the amount of time spent on a point in time recovery? I'm pretty sure I know the answer to this, but the limitations of a point in time recovery blows my mind.

Say I have a 2TB database set to FULL recovery mode. I diligently take weekly full backups, daily diff backups, and log backups every 30 minutes.

Now, I want to roll the database back to 1 hour ago. Do I really have to restore the entire full backup, diffs, and logs up to one hour ago? Am I correct that this will take a very long time?

Is there no way to ROLLBACK the changes using the transaction log files so that this type of recovery takes a few minutes?
Yes, you would have to restore the full backup, at maximum one diff backup, and all necessary log backups to perform a point-in-time recovery in SQL Server. In SQL Server, you cannot roll back already committed transactions to go back in time. You would likely want to use the STOPAT syntax to restore your database to the exact desired point in time. 

In fact, because you are doing transaction log backups every 30 minutes, SQL Server is also clearing out the transaction log with each log backup--so to go back to an hour prior, SQL Server wouldn't even have the necessary information to roll back that far, even if SQL Server had such a feature (which it doesn't, sadly).

I can't answer whether it would take a "very long time" because restore time varies quite a bit based on how large the database is, how many changes are contained in the log / diff backups, and how many log backups need to be restored. For a small database that isn't very busy, the restore could be quite fast, but for a very busy multi-terabyte database, it could take a while.

Oracle has a feature called Flashback that will let you do what you describe, but SQL Server doesn't have an equivalent feature.