How to keep a component "pressed" so the chain remains hooked?
Yes, that is the rear derailleur. FYI there is a Terminology Index on this site that provides all component names.

It seems the problem you are having is that the derailleur is not lining up with each of the rear sprockets, so the chain is not running on them properly.

Most modern derailleur systems are indexed, that is the shifter lever has discrete clicks that correspond to each sprocket. What has probably happened on your bike is that the cable has stretched or loosened so that the derailleur does not sit directly over a sprocket at each position.

The derailleur or shifter cable usually has a barrel adjuster that can shorten of lengthen the cable to adjust the dearilleur position. In your case the cable likely needs to be detached from the derailleur and the effective length reset.

There are several online guides that detail how to adjust derailleurs. This one from Park Tool is probably one of the best ones - https://www.parktool.com/blog/repair-help/rear-derailleur-adjustment.