In this paper, we propose a three-party and a multi-party quantum key agreement protocols with single photons in both polarization and spatial-mode degrees of freedom. Based on the defined collective unitary operations, the participants can agree on a secure shared key through encoding their sub-secret keys on the particles. Moreover, the security of our protocols is discussed comprehensively. It is showed that the presented protocols can defend both the outside attacks and participant attacks. The efficiency analysis also shows that our two protocols can achieve high qubit efficiency. Besides, our protocols are feasible since the preparation and themeasurement of singlephoton state in both polarization and spatial-mode degrees of freedom are available with current quantum techniques.