Is there a difference between private key cryptography and symmetric encryption? I have often seen these terms being used interchangably, but is that correct? Are there Private Key Algorithms that do not use Symmetric Encryption?
Private key cryptography is often used as synonym for symmetric cryptography, that is, cryptography with constructs that use a symmetric key. For instance professor Katz uses this terminology.

Symmetric keys can be used for e.g. a block cipher such as AES, a stream cipher such as RC4. HMAC - which is a keyed hash construct - also uses a symmetric key, and doesn't use a cipher at all. The term "symmetric encryption" is usually limited to constructs that provide confidentiality, although authenticated ciphers may also provide integrity and authenticity of the encrypted messages.

So the question " Are there Private Key Algorithms that do not use Symmetric Encryption?" has the response "yes" for the simple reason that HMAC it does not provide encryption, and it is certainly a private key algorithm. Symmetric encryption which is a goal rather than a construct uses certain private key algorithms to accomplish the goal.