Which parts of the cerebral cortex don't belong to the neocortex?
The hippocampus is considered archicortex rather than neocortex. Sometimes hippocampus is considered "sub-cortical" but I think this is only really fair if you really mean "sub-neocortical"; it's common for "cortical" to be used as a shorthand for neocortex in human neurobiology. Writers will be more specific when necessary.

The piriform cortex and related olfactory structures are considered paleocortex.

Some also delineate an in-between periallocortex that includes entorhinal cortex and limbic structures around the hippocampus.

Neocortex is mostly a six-layered structure and found only in mammals. The other, non-neocortical types of cortex have fewer layers, and are shared by an older common ancestor that includes other vertebrates.