Interfacing with the kind of LCD found in thermometers
Your low-cost option at hobby quantities would be a serially interfaced numeric LCD part such as the ModuleHouse KTM-S1201 12-digit display, available for under $1 in single quantities. 

This particular device is popular especially among RC DIY hobbyists. It consists of the actual LCD panel (datasheet) driven by an integrated NEC µPD7225 programmable LCD controller (datasheet) (updated application note).

There are some handy notes on how to interface with the device here.

Libraries to control this device have been posted, including one written in C here.

Besides displaying the hexadecimal digit set (0-9 a-f), 7-segment displays can be used to display, albeit crudely, several other letters and symbols. This particular device can be set to an individual segment addressing mode if you want to display characters additional to the hexadecimal digit set. For instance, look at the left half of the image below:



There are also 7-segment based serially controlled LED displays such as this one on SparkFun, if you prefer.



To address the bullet points in your question:


Driving an LCD display involves multiplexing, contrast management and character set translation, hence, yes, typically an LCD display driver IC is used for this purpose. The µPD7225 is one such IC.
Shift registers are an acceptable method for driving 7-segment LED displays, but not so advisable for LCDs
The more direct method of using a serially controlled display product, as described above, works out most cost and effort effective, compared to fudging around with various ICs and passive parts to get to a simple solution.
There are several microcontrollers which are designed to drive LCD panels directly. For instance, the Texas Instruments MSP430x4xx family. Cost and effort are higher than for the ModuleHouse part, but if you are keen to experiment, that may be an avenue to follow.


I hope this addresses your question sufficiently.