Finding the difference between the estimated value and the true value in a reward task
Using list comprehensions instead of numerous .append calls is
often times faster and more readable.

A list of rewards should be called rewards instead of reward. Same goes for values and true_values.

We don't need to enumerate rewards, since we don't need the index itself. We can simply iterate over rewards.
def loopfunction(alpha, rewards, true_values):

    values = [0.5]
    for reward in rewards:
        values.append(lever_update(alpha, values[-1], reward))

    values.pop(0)

    return [abs(value - true_value) for value, true_value in zip(values, true_values)]



I sadly can't test performance right now, but I would assume the first for loop takes up most of your runtime. This is what I'd focus on for further improvements. In general, making code more concise and readable also allows for easier refactoring / performance improvements in the future.

We can also make lever_update more concise:
def lever_update(alpha, value, reward):
   return value + alpha * (reward - value)