What is retrieved from disk during a query? Fairly simple question, probably answered somewhere, but I can't seem to form the right search question for Google...

Do the number of columns in a particular table affect the performance of a query, when querying on a subset of that table? 

For example, if table Foo has 20 columns, but my query only selects 5 of those columns, does having 20 (versus, say, 10) columns affect query performance? Assume for simplicity that anything in the WHERE clause is included in those 5 columns.

I'm concerned about the usage of Postgres' buffer cache in addition to the operating system's disk cache. I have a very lose understanding of Postgres' physical storage design. Tables are stored across several pages (defaulting to 8k in size per page), but I don't quite understand how tuples are arranged from there. Is P.G. smart enough to only fetch from disk the data that comprises those 5 columns?
The physical storage for rows is described in the docs in Database Page Layout. The column contents for the same row are all stored in the same disk page, with the notable exception of TOAST'ed contents (too large to fit in a page). Contents are extracted sequentially within each row, as explained:


  To read the data you need to examine each attribute in turn. First
  check whether the field is NULL according to the null bitmap. If it
  is, go to the next. Then make sure you have the right alignment. If
  the field is a fixed width field, then all the bytes are simply
  placed.


In the simplest case (no TOAST'ed columns), postgres will fetch the entire row even if few columns are needed. So in this case, the answer is yes, having more columns may have a clear adverse impact on waster buffer cache, particularly if the column contents are large while still under the TOAST threshold.

Now the TOAST case: when an individual field exceeds ~2kB , the engine stores the field contents into a separate physical table. It also comes into play when the entire row doesn't fit into a page (8kB by default): some of the fields are moved to TOAST storage. Doc says:


  If it's a variable length field (attlen = -1) then it's a bit
  more complicated. All variable-length data types share the common
  header structure struct varlena, which includes the total length of
  the stored value and some flag bits. Depending on the flags, the data
  can be either inline or in a TOAST table; it might be compressed, too


TOAST'ed contents are not fetched when they're not explicitly needed, so their effect on the total number of pages to fetch is small (a few bytes per column). This explains the results in @dezso's answer.

As for writes, each row with all its columns is entirely rewritten on each UPDATE, no matter what columns are changed. So having more columns is obviously more costly for writes.