Must you work with all qubits in a circuit when applying a gate?
You can apply a quantum gate on any subset of the qubits, not affecting the rest.

This can be formalized as evolving certain qubits via $U$ while the other qubits evolve via $I$, thus the joint unitary is $U \otimes I$.