How digital signatures work
The article is oversimplifying matters. In a real-world scenario, either…


The recipient already has a copy of the public key, or can retrieve a copy from a trusted source. Either way, they can confirm the key is correct by comparing it with the "known good" copy they already have. (This is roughly how PGP works.) Or…
The key has a signature attached to it, signed by a trusted central authority. (This is roughly how SSL/TLS works.)


Either way, your analysis is entirely correct -- attaching a signature and public key to a message doesn't confirm anything on its own.