Least populated time-zones
There is one timezone that is completely uninhabited: the UTC-12:00. It covers only two islands: Baker Island and Howland Island that are both uninhabited (owned by US). Because of the weird way timezones are divided in the pacific there are timezones that practically use the same time (e.g. UTC+12 and UTC-12), but are actually in two different days. However if you strictly interpret the question no-one is lonely here as there is no-one here, so this might not comply.

The second least population is at UTC+08:45 that covers village of Eucla and other villages in Australia with a population of around 200. The area is surrounded by signs to remind you to turn your clocks, but it is unofficial so it might not comply either.

The third least population is at UTC+10:30 that covers an island called the Lord Howe Island that belongs to Australia and inhabits 382 people. If you don't accept timezones that aren't in integer changes from UTC then this UTC+10:30 might not satisfy your curiosity.

The first full integer timezone shift from UTC (that has any population) is UTC-02:00 that consists of Atlantic islands of Brazil like Fernando de Noronha, Trindade, Martim Vaz etc. Also South Georgia and South Sandwich Islands of UK belong in this timezone. The total population then is somewhere around 2700 people. Unfortunately these are the places that use this timezone all year round and other areas like densely populated Brazilian cities use this timezone as daylight saving during their summer increasing the population drastically so even this timezone might not satisfy your criteria.

Finally UTC-11:00 being full integer change from UTC, with only all year round usage (isn't a daylight saving timezone for anyplace) consists of American Samoa, Midway Atoll, Jarvis Island and Palmyra Atoll of US and Niue of New Zealand and has under 60000 inhabitants.

Ps. The estimated population numbers are from the wikipedia pages of the locations mentioned.