What technical properties does a confirmed Tx have that an unconfirmed Tx does not? From a user perspective, each block confirmation reduces the probability that a transaction payment will be reversed.

However from the perspective of the software, what happens when a transaction is confirmed? For example, I know it is removed from the mempool. Is there any other technical difference between confirmed and unconfirmed transactions? Are both outputs stored in the same UTXO set? Does anything happen on any lower level (data storage, etc) inside a node or on the network or in the protocol once a Tx is included in a block?

Do confirmed transactions have any special properties besides the human perception of safety?
when we speak about specific transaction: no. The unconfirmed transaction looks exactly the same as the same transaction with 1,2.. confirmations.

yes. There are 3 types of transaction: confirmed, unconfirmed, and nonexistent

example of nonexistent are those 10BTC I'll give you tomorrow.

difference between unconfirmed and nonexistent is not "obvious"

When you see an Unconfirmed tx on your PC, this does not "guarantee" that everybody else sees it too. If internet connectivity fails, you may be precisely the only person who sees it.

Unconfirmed transactions can be very different from confirmed, in many ways:

Unconfirmed transaction can spend funds that already have been spend, or funds that never existed.

Unconfirmed transaction get lost in the Internet (never get confirmed).

Unconfirmed transaction can be reversed and be replaced by different Unconfirmed transaction sending the funds to somebody else.

Unconfirmed transaction can contain technical error that prevent it to be confirmed.

Unconfirmed transaction can have low fee (less than dust limit) and this can cause confirmation delay of 12 or more hours.

Unconfirmed transaction can be one of many abusive (spam) transaction (in an attack). Such Unconfirmed transactions will typically never get confirmed and most likely all miners will delete these abusive transactions.

Unconfirmed transaction can be censored by all miners. This can happen if some address belongs to a person who is anti-bitcoin or some large corporation that has a conflict of interest with bitcoin (like a large bank). If all miners agree that they do not wish such entity to transact on bitcoin, they may decide to censor the activity of such an entity or person. This transaction may never get confirmed.