Does Finn Edwards have to pay to upkeep his Bank Loan?
That's absolutely correct (although there's also the additional penalty to the end-game scoring if you do that thing). You can always choose to default on a bank loan and lose all of your items. Since Finn doesn't have to discard items, he gets to keep everything. Basically, the Bank comes a-knockin' and all of his stuff is stashed away. It seems to me like this is just the situation a Holdout would be wanted for.

Now, you could choose to interpret the 'pay $1 or discard' as a cost rather than a loss, which would prevent Finn's ability from protecting him. Much like the 'discard' section of the dynamite or the holy water is a cost of using the item, I could see an argument for the 'cost' of a bank loan being either paying $1 and repeating the process next turn or losing everything. But I've always seen it played that Finn's ability protects him from the Bank Loan.

The same question was asked here in the Kevin Wilson answers thread on the FFG forum (although the question wasn't answered by Kevin)