SQL Server Partition Switch
Not sure what you mean by "manage"... one of them has to wait for the other. 

Imagine you need to change your tires or get an oil change, but your son took the car to the mall.

That is what is happening here. The metadata change has to wait for the data change to commit. You can control which has higher priority, though, by adding a wait option to the switch:

WITH 
(
  WAIT_AT_LOW_PRIORITY
  (
    MAX_DURATION = n MINUTES, 
    ABORT_AFTER_WAIT = BLOCKERS 
  )
);


At the specified timeout you can choose to kill the blockers (BLOCKERS) or die gracefully (SELF). Number of rows in the partition is largely irrelevant (unless you are waiting for ~12M rows to actually update). 

See Kendra's post here (do look through the comments) and this background post on low priority.