Generating public/private RSA key pair programmatically, using it for JWTs
There isn't too much to see here because the key generation simply relies on RSA.generate(2048), but I wonder why you would need this code as it is exceedingly shallow.

Regenerating key pairs for signing at startup is utter nonsense because a key pair is next to useless if the public key isn't trusted by the receiving party.

Exporting the private key as unprotected PEM is also very dangerous. The method would fail if hardware support (a HSM) is ever used.

The inclusion of  payload.update(dict(iat=datetime.utcnow())) is undocumented. Although it may be expected for webtokens, a caller will want to know that this is performed. Undocumented side effects are a sure fire way of breaking the principle of least surprise.

Passing kwargs** without any validation doesn't give me a very warm feeling inside.

Slightly less of an issue maybe, but an RSA 2048 bit key provides a key strength of only 112 bits or so. You may want to consider at least 3072 bits.

This code is clearly related only to webtokens and RSA keys, even with a specific key size. So allowing an argument algorithm: str='RS256' may just allow a user to get the object instance in an invalid state, and not much more.