How can we prevent AGI from doing drugs?
You make a lot of assumptions about AGI, namely that 'we need a computer to compute the utility and reward AGI'. It not clear to me that (1) we can achieve AGI, (2) AGI will be on a computer as we know it and (3) AGI will work with a utility / reward function as we know them.
One thing I am sure though is that ML is known for &quot;cheating&quot; (see for exemple). Avoiding such cheating is part of the building process. So, when you assume that we can achieve AGI, it means that you assume we can build an AGI that do not &quot;cheat&quot;. Thus, the answer is mostly contained in your assumptions.
Whether we are able to build an AGI, what &quot;cheating&quot; we would have to overcome to do so and how we will be able to do so is mostly undetermined.