Is it possible to install a 1x11 drivetrain on an old MTB?
I did half of this - my late 80s 3x5 speed freewheel with nuts got upgraded to a 3x7 freehub with quick release, because the wheel was available.  Turned out the freehub was actually 8 speed, so I could have gone as high as 3x8.

However its already hard to get the wheel into the frame - I don't think you'll fit a wheel and an 11 speed cassette in there.

Also the QR has to be checked after every ride.  I've had the back wheel rotate slightly in the dropouts and it rubs on the chainstay, which means it moves at the power stroke.  I did not cold-set the frame, so the dropouts are not parallel | | they are now slightly \ / which allows a little movement and eventually the QR comes loose enough to move.  Depending on conditions this might be 10 to 100 km of riding to back off loose.

The rear thumb shifter will need to change, that's a given.  I got away with using the same rear deraileuer mechanism and wire, but had to change the lever to a 7 speed indexed one.

You'll also need to change the chain.  5 speed chains are quite thick and may bind up in a 6/7/8 cassette.

Mine cost $50 for a cassette, $3 for some spacers, $40 for the thumb shifter, and another $40 for the new 6/7/8 speed chain.   That was probably too much, but the freehub and chain needed doing anyway.

Summary  this is probably not worth doing unless you get the replacement parts really cheap (ie free or almost free.)