What is the best practice to upgrade Drupal core without breaking the Git repo and the production website I've got a Git repository in which all my code is in the master branch, and I was previously just ignoring all the Drupal files, so that I kept a strict separation between code that I wrote (or modified or might modify) and code that could be generated with Drush or whatever.  

This seemed like a good strategy until I had to upgrade Drupal.  I realized I want to be able to roll back if things went poorly, and what better tool to use than Git to do that.   I thought to myself this would be the perfect situation for a feature branch, so I made a drupal-7.14 branch, gave it its own .gitignore to ignore all my code and settings files and pay attention to only files that are part of the Drupal install which I wouldn't be touching.  I did an upgrade by hand (download, unzip, untar, copy), sorting through borderline cases like robots.txt and .htaccess, and overwriting Drupal's .gitignore with my own.  I fixed some settings that worked with 7.14 but not with 7.15, to recover from a 500 error, and then everything seemed perfect.  I renamed the branch to drupal-7.15 and was about to go happily on my way.

Until I realized what I had inadvertently done:  files that were previously untracked by my master branch but left in the working directory were now removed from the working directory when I checked out master, since they were no longer untracked files!  D'oh!

If I merge the drupal-7.15 branch with master I'll lose the separation of code. 

There's probably some way to convert a branch to a submodule.  Assuming that's possible, that might be the best strategy.  I knew before I did this that submodules were the "right" solution, but since I didn't realize the side effect of using branches for previously untracked files, I decided to cut corners and go that route.  (Also, all the approaches I've seen to using submodules with Drupal assume that you're starting a new project and Drupal will be the master branch.  It is undesirable to me to make someone else's code the master branch, and I already had a repo with a master branch.  This looked like it would be unnecessarily complicated just to do an upgrade.)

There may be some other solution that I haven't thought of.

How can I best recover from this with the fewest possible downsides?

UPDATE: This is in development (in a Linux VM on my laptop), and has not gone to production yet.  By the time we go to production I plan to have everything wrapped up in feature modules, but that's not in place yet.

UPDATE 2: Submodules may not work.  According to Pro Git, "Submodules allow you to keep a Git repository as a subdirectory of another Git repository".  Drupal doesn't provide any such a nice separation.  Instead of all Drupal code being in a subdirectory, the relationship is more or less inverted, but there's still no clean separation, since you may be editing your .htaccess and robots.txt, so your code and the Drupal repo are mixed together.  I'm looking for a workaround to this problem.
It seems from discussion above that your question isn't about fixing your git repo, but about maintaining a Drupal site in git, and how that works with core updates.

I think that standard practice is in fact keeping all files in your repository, including Drupal core. The separation of Drupal code from custom code seems like a nice idea, but I don't think it's necessary, and I don't see what advantages it gives you. It also seems to assume that you will never want to patch core (or have to do it as a part of your deploy), which, while it isn't best practice, is definitely something you want to be able to do if something breaks in production. 
Keeping your commits nice and focused also helps to get this kind of separation.

The solution I've used is to keep all code in the same repository, put as much as possible in Features or other kinds of exports/code/configuration, and maintain a list of patches that need to be applied to out-of-the-box core and contrib.

When you update core, start in your dev environment:


Make sure working directory is clean
Dump latest database (drush sql-dump). Either make a commit with the dump or save it somewhere safe.
Update core (drush up drupal)
Commit all changes. 
Check patches file. If any patches need to be applied, apply them, and make another commit
Run update.php if necessary (already done if you used drush for the update)
Test your dev site. If everything is fine, push code to production. Run drush updb on production.
If there's a problem and you need to revert, either revert or reset your repo (git reset --hard HEAD~2 should do it), or revert the two commits if you want to keep the history. Replace your database with the dump.


The database dump is necessary because you otherwise can't undo changes made to the database by updates. You can also do the updating in a branch, in which case reverting just means checking out master. It's slightly inadvisable if you're working on one dev site because you can't really switch back to master and keep working there in parallel because of the database.

Using this more simple git-side workflow will allow you to use git's full power when you need it without the complication of submodules, etc. If this doesn't seem adequate, could you explain why you need further separation of code?