Watch 4 buttons and certain features frequently go unresponsive
I'm going to assume there's no pattern to when this happens and it's totally random.

My first thought would be that it's a connection issue (remembering that your watch is designed to communicate with your iPhone). So I'd go through the process of unpairing and then restoring your Apple Watch to see if that resolves the issue. 

IMPORTANT: - Read through all of this answer before commencing the process so that you know what's involved.

You can un-pair your Apple Watch from the Watch itself or from your iPhone. Since you're having frequent unresponsiveness from your Watch, the following steps are for doing this from your iPhone.

Un-pair and restore your Apple Watch


On your iPhone go to the Apple Watch app
Make sure you're on the My Watch screen
Tap on your watch name at the top of the screen
Tap on the info icon (i.e. the amber circle with an 'i in it)
Now select the red Un-pair Apple Watch option
You'll be prompted to confirm the un-pair process
If you have an e-sim activated with a mobile service, choose to keep your plan
Disable Activation Lock by entering your Apple ID password and tapping Un-pair
Your watch data will begin backing up to your iPhone
When complete, your watch will:


be unpaired from your iPhone
be unpaired from any Bluetooth devices
no longer have any Apple Pay cards attached
have no passcode
be reset to Apple's factory settings

OPTIONAL: You may want to backup your iPhone as it's now the only place all of your Apple Watch data is stored
Switch on your Apple Watch
Put it on
Now make sure your iPhone is close to your Apple Watch
Once the "Use your iPhone to set up this Apple Watch" prompt appears, tap on Continue. (NOTE: If the prompt doesn't appear, open the Apple Watch app on your iPhone and tap the Start Pairing option)
Follow the prompts to pair your Watch to your iPhone and to set it up


Now start using your watch again to see if it works properly. If it doesn't I'd call Apple Support. They can actually run diagnostics on your Apple Watch remotely (you will need to have your iPhone with you as well in order for them to do this). It doesn't take long to do and will help identify if there are any hardware issues.