What are the downsides of a pressurized room?
In a properly configured home, the return air ducting is the exit path for airflow. There's no significant pressure buildup in any room regardless of door position. 

In older homes without return ducting, or vents solely in common areas, the return air path is indeed through or around the door. Still, pressure buildup is extremely small unless the door is extraordinarily well sealed.

You should probably ask "someone" what his or her concern actually is. The pressure differential in your situation is miniscule--probably not even enough to pop your ears. The only downside I can come up with is the accumulation of dust bunnies outside your door.