How to remove unreferenced rows in a "Many to One" relation?
A Person and a House must be associated with a City, but a Person does not necessarily have to be associated with a House. If a Person is associated with a House, they must be associated with the same City.

Your second diagram is correct. Both Person and House are independently associated with City. You just need another table.

Simple, you create a type of intersection table to relate a Person to a House that also forces Person and House to refer to the same city.

create table PersonHouse(
    PersonID int not null,
    HouseID  int not null,
    CityID   int not null,
    constraint PK_PersonHouse primary key( PersonID ),
    constraint FK_PersonHouse_Person foreign key( PersonID, CityID )
        references Person( ID, CityID ),
    constraint FK_PersonHouse_House foreign key( HouseID, CityID )
        references House( ID, CityID )
);


PersonID is the PK so a Person cannot reside in more than one House but the same House may contain more than one Person.

When inserting a record into PersonHouse with Person 15 who (in Person table) is associated with City 37, the House ID must be contained in the House table and must also be associated with City 37. Any attempt to enter a Person and House associated with different cities will generate a foreign key violation.