How does Australia fit into 'New World' / 'Old World' dichotomy?
First off - this is an answer without citations. I looked for them, but couldn't find any. The terminology is undoubtedly described in reputable sources, but it is unlikely a paper is dedicated solely to explain this terminology, making literature searches tricky. Having said that -  

I discussed this question with an anthropologist and she agrees with my idea that the terminology Old and New World is a historic and Euro-centric one, dating back to the colonial times when Europe viewed itself as the center of the world. 

The Americas were discovered relatively early, around 1492. It was designated the New World. Australia wasn't found until around 1606. By that time the New World was already a century old and very likely already solidly associated with the Americas. 

The New World, in a biological context, is often used to describe groups of closely related species found in the Americas and not elsewhere, such as New World monkeys. In certain areas of expertise the terminology is baked in in the jargon, .e.g,  monkey and ape behavioral research areas. New World monkeys can be referred to by the Cebidae, as per the Linnaeus's classification. However, due to practical reasons geographical clustering of species can be more convenient.

The term New World means different things in a biological context or in the context of wines, for example (Americas versus Americas + Australia - admittedly Australian wines are pretty good). 

In all, I think the terminology 'New and Old World' is confusing, rooted in historical considerations. Australia as well as Antarctica don't fit in this terminology.