Transaction fee required by bitcoin-cli keeps increasing as I increase the fee
The problem with sendtoaddress is that the transaction size isn't known in advance.  It may be that by adding a fee, you are needing to include a new input in the transaction, but inputs are the largest and therefore the most expensive part of the transaction, so that makes it still more expensive.  That would be most likely if you have a lot of addresses with small amounts in them.

Also, settxfee sets the fee per KB, but from a quick review of the code, that message seems to be printing an absolute amount rather than a rate per KB.

You may want to adjust the settings described here:

https://bitcoin.org/en/release/v0.12.0#wallet-transaction-fees

For complete control, you can use createrawtransaction/fundrawtransaction, or use an external system (e.g., coinb.in) to create the transaction and just sign it using your core wallet.