Are UTXOs signed with a private key?
That's correct.
UTXOs are just entries with the following fields:

txid of the transaction that created it
output position in that transaction that created this output
the output itself (consisting of scriptPubKey and amount)
the height of the block that created this output (needed for relative locktime checking, see BIP112)