What should I look for when shopping for a kitchen faucet water filter?
The first question to ask is what you want to filter out and why. From the wording of your question, it appears that you think your drinking water is unclean. As far as I know, public drinking water is safe everywhere in the United States, so I'll assume that you have well water. If so, you need to start by having the water tested. Use a local or semi-local lab, as they will know what to test for in your area. The staff of nearby public water districts can probably recommend a reputable, local lab.

Depending on the test results, there are different kinds of filters. If your water is very hard, some filters will require that you also have a water softener.

If you are just looking to improve the taste of safe water, then a carbon filter will likely suffice. They can be mounted on the tap; alternatively, a whole-house filter can be installed for a few hundred dollars. (That may not be an option for an apartment dweller.)