Understanding need for PKI For simplicity let's say I only have RSA and AES on my hands and I want Alice to talk with Bob over a secure channel. With secure I am implying no MITM and Alice and Bob know they are talking to each other and not someone else.

When Alice wants a secure channel with Bob, Alice will ask Bob for his RSA public key, sign a randomly generated AES key with her RSA private key, encrypt the signature and AES key with Bob's RSA public key and send this to Bob. Bob will ask Alice for her RSA public key, will decrypt the signature and AES key with his own RSA private key and verify the signature with Alice's public key. Now Alice and Bob can use the AES key for further communication ...

Am I correct that the part where Alice and Bob exchange their public keys is a potential MITM vector? How can Alice be sure she got Bob's public key and vice versa? Is this what PKIs primarily solve, as in Alice and Bob trusting a third party upfront that - for example - signs their public keys? Or am I missing the point completely and everything of the above is bogus?
Your understanding is correct. PKI would be the service that verifies that Bob's public key really belongs to someone named "Bob". As long as Alice trusts the certificate authority that certified Bob's public key, she can implicitly trust that the public key she sees really is Bob's, not Eve performing a MITM.

The internet is secured with web PKI, which is the network of Certificate Authorities that certify that a websites' public key really belongs to the domain it claims to belong to (for Domain Validation, or DV). It's far easier to trust a handful of CAs than it is to trust every single website you'll ever visit.