Which machine learning model should I learn for this problem? I'm working in python. Would like to practice some machine learning, and I've always been curious about an analog to the problem below...

A collection of 3 letters are drawn randomly from the 26 letters of the alphabet. None, any, or all of the letters can be discarded and replaced with an equal number of not-yet-drawn letters. This discarding and replacing happens 3 times (or more generally n times). After all the replacements have taken place a point value is awarded to the resulting 3-letter-word. Some words have high point values, and others less so. I'm interested in the optimum replacement strategy. For example, say only the words "FOX" and "THE" reward any value. All other 3 letter combos are worthless. I want the machine to learn the correct replacement strategy while holding "FOW" with one replacement remaining. In this simple case, the strategy is to replace the W only, and attempt to draw the X. This strategy is superior to replacing all 3 letters in an attempt to make "THE," since drawing "THE" only gets points 1/('count of remaining letters' choose 3) times, whereas replacing the W gets points 1/('count of remaining letters'); the instance where an X is drawn.

Can anyone point me in the right direction?
You should use a Markov reward model to model your problem. All the possible words are the different states of your chain. The replacement process corresponds to the transitions of your Markov chain.

After defining all the properties of your chain (states, transitions, rewards, ...), you can train your model and get the best strategy for each current word.