NVENC_HEVC Encoding making my files larger
Just because you can get a hamburger at different restaurants doesn't mean they'll taste and look the same.

Apples and oranges

x265 and NVENC are two different encoder implementations that can output the same format, HEVC. What settings, options, and algorithms they use is up to the authors as long as the output conforms to the specs.

Choose the right tool for the job

These encoders are generally for different use cases. x265 may be slower, but it provides more quality per bit. NVENC is more suitable when speed is more important than quality or encoding efficiency, or when the CPU is not available or not up to the task, such as on a mobile platform where using the CPU would possibly consume more battery power and take a long time.

Option mixup

Also, you're mixing up options: -rc vbr_2pass -rc-lookahead 20 -gpu any are private options for nvenc_hevc, but you attempted to apply them to libx265, so they were ignored. There are two types of options: general and private. The private options are specific to each individual encoder. You can list private options such as with ffmpeg -h encoder=libx265.

You'll have to experiment with the encoding presets (see -preset option for both encoders) and rate control methods to get an output that is acceptable for you.

Waste of time?

Lastly, unless your input H.264 files were encoded very inefficiently (or are lossless which is doubtful), or if storage space is of utmost importance, then re-encoding from H.264 to H.265/HEVC is a waste of time and effort.