Simple Definition of a Simple Grammar (with confusing example)
You're right, the grammar is wrong. Everything would be fine, though, if you were able to eventually eliminate the $B$. Here's a way to do that:
$$
S\rightarrow b\mid aA\\
A\rightarrow aB\\
B\rightarrow aB\mid b
$$
The first production generates a singleton $b$ or the first $a$, the second generates the second $a$ and the third eventually generates an arbitrary number of $a$s followed by a $b$.