Combine audio and video file without creating a 3rd file
I found a solution to my problem:

Ffmpeg can use a http stream as input:

ffmpeg -i YoutubeVideoOnlyURLHere -i YoutubeAudioOnlyURLHere -c copy -map 0:0 -map 1:0 Path/to/outputFile.mp4

You can get the needed URLs from
youtube-dl -f bestvideo,bestaudio --get-url https:www.youtube.com/watch?v=XXXXXXXXXXX