How to create vectors from text for address matching using binary classification?
I would rather recommend to consider each address as a separate instance. (Separate row in feature matrix)

Each address can be further broken into attributes like: House no., Street, City, State, Country.

Above attributes can be vectorised based on distance between them. Like vectorise each country from west to east(incremental numeric value). Then vectorise each state in country from west to east. Similarly for street. In case two entities fall on same longitude, assign consecutive numerical values.

Once each attribute is vectorised, similarity between two addresses can be Euclidean distance between vectors of two addresses.

Anyways leaving aside data science approach, this problem can be solved further simply by getting distance between two addresses using Google Maps API.