Are render passes always rendered?
The contents of some render passes is always computed as part of rendering the combined pass. It takes little extra time to allocate the pass and store that value, however for a simple render and many passes this overhead will add up too.

For others (e.g. the Motion Vector pass) it would not normally be computed, so it will take up more time.