What are the right post-installation procedure after installing git using Homebrew?
You should at least create a file called .gitconfig and possibly also one called .gitignore_global in your home directory.


.gitconfig:
specifies some default settings; at the very least:

[user]
    name = your_name
    email = your_email_address
[core]
    excludefile = path_to_your_home_folder/.gitignore_global



.gitignore_global:
files you always never want to track / put under version control in any of your projects; mine looks like this:

.DS_Store
.localized
*.swp
*.pyc


The first two are files, which OS X might put into a folder; *.swp files are vim swap files and *.pyc are precompiled Python scripts. 
After that you should be ready to go.

The "caveats" are actually information, not really warnings:


git-credential-osxkeychain is a tool that helps you managing your GitHub credentials; the "caveat" simply tells you where you can find the tool
the contrib folder contains stuff that "others" have added to git; in my installation the folder only holds the "shell completion" code that is mentioned in the next two caveats you got at the end of the installation; again this is merely an information where you can find the contrib folder
the "shell completions" mentioned in the last two caveats provide code to "tab complete" the names of branches and tags when you run git commands in your shell of choice. This is similar to how you can "tab complete" file and folder names in the shell. Once again this is just an information not a warning.


So, long story short: these "caveats" simply provide additional information about your git installation, but you do not need to do anything about them to make git work.