Should I be using node_load() rather than building my own queries with db_select()? I'm fairly comfortable with using Drupal 7 functions like db_select(). While node_load() is obviously very easy to use, I often find that it returns fields I don't need and doesn't provide fields that I do need (field that would require some joins to obtain).

Therefore, will building my own queries provide better performance than using node_load() in conjunction with other queries? Or is there some kind of performance boost that results from using node_load(), perhaps from cacheing, etc.?

I'm far from an expert on MySQL performance issues, but it seems to me that constructing a single, specific query is better than using multiple queries that individually only retrieve part of what I ultimately need (and a lot of what I don't).
Technically, yes, writing your own db_query() statements will return the data faster and with less overhead. However, it is not really the "Drupal way" to do things.  

node_load() provides a 'static' cache; in other words, if you load the same node twice in one page request, it will only hit the DB once.  

What you are really missing out on by not using node_load() is all the API hooks and third-party module integration. If you need a CCK field, node_load() gets all that data for you, where as a manual query would require a lot of joins.  

As you note, there is a lot of extra processing/loading going on that you may not need at the given moment. If you don't need the data, it won't technically hurt you not to have it. It is really up to you if you want to go this route.  

The biggest reason IMO for using node_load() even though you might not need the extra data is just for the purpose of standardization. You won't have to worry about what pieces of data you have at a certain point. You will have a fully loaded node and will have more options for adjusting the data as necessary through hook_nodeapi(). Other developers looking at your code will have an easier time understanding what they are working with. Also, 6 months down the road when you go to add a new feature/display field you wont have to adjust your queries.  

node_load is definitely a resource hog, but i have found it better to use in my experience.  Drupal 7 greatly improves the situation by providing node_load_multiple() which, as its name implies, reduces the number of queries when loading multiple nodes.