Are UTXOs signed with a private key? Are UTXOs signed by a private key?
I'm trying to understand exactly which parts of Bitcoin use public and private keys. You need a private key in order to generate transaction output, and to create a transaction that uses those outputs, but I'm not sure what happens in the middle.
My current guess is that they aren't signed. The transactions generating the UTXOs are signed, and the block containing the transactions can be validated and then traced back to the miner. It doesn't seem necessary to sign the UTXOs, and even if we wanted to, the only private key available would be the miner's.
That's correct.
UTXOs are just entries with the following fields:

txid of the transaction that created it
output position in that transaction that created this output
the output itself (consisting of scriptPubKey and amount)
the height of the block that created this output (needed for relative locktime checking, see BIP112)