1. A method for manufacturing a semiconductor device with group III-V channel and group IV source-drain, comprising: preparing a substrate, the substrate selected from one of the group consisting of a Si substrate, a Ge substrate, a Si substrate with Si x Ge 1-x (x=0˜1) or GaN or silicon germanium carbide grown thereon, a Ge substrate with Si x Ge 1-x (x=0˜1) or GaN or silicon germanium carbide grown thereon, and a diamond substrate with Si x Ge 1-x (x=0˜1) or GaN or silicon germanium carbide grown thereon; depositing a dummy gate material layer on the substrate and defining a dummy gate from the dummy gate material layer by photolithography; doping the exposed region of the substrate by self-aligned ion implantation using the dummy gate as a mask and activating the exposed region at high temperature, so as to form source-drain; removing the dummy gate; forming a recess in the substrate between the source-drain pair by etching, the recess having a depth required for forming a channel-containing stacked element by subsequent epitaxy; forming the channel-containing stacked element in the recess by epitaxy; and forming a gate on the channel-containing stacked element.