text: This is the mail archive of the systemtap@sourceware.org mailing list for the systemtap project.
  This is all about maintaining the systemtap packages in Fedora. If you aren't interested in helping with Fedora maintenance, then there is nothing about systemtap here to follow. This week has been some mayhem in Fedoraland, converting from dist-cvs to dist-git. You should follow devel@lists.fedoraproject.org for general details about the whole new world order for package maintainers. You'll want to start with: # yum upgrade --enablerepo=updates-testing fedora-packager Versions before 0.5.1.0 don't work. (I presume that by next week sometime Fedora updates proper will have a good version.) If you have an old CVS checkout from /cvs/pkgs for systemtap, then go look at it. The CVS servers are still there for read-only use, so you can do a cvs update and make sure there is nothing in your old checkout that you wanted to keep. Start in the new world with: $ fedpkg clone systemtap This is actually just doing: $ git clone ssh://pkgs.fedoraproject.org/systemtap but you don't have to remember the URL. The new scheme is the same as the old scheme in essence: the .spec file, any patch files, and a "sources" file (md5sum list) are in this repository. Things you previously did with make targets are now done with fedpkg subcommands, mostly of the same names as teh old make targets. It's all very much still being worked on, so go talk to Fedora folks on mailing lists or IRC (#fedora-admin) if you hit problems. 'fedpkg help' will get you started, and each subcommand has a --help. Unlike the old dist-cvs scheme, each package (i.e. systemtap) has just one actual repository and the different maintenance streams are on normal proper git branches. This is what I want to tell you about in a little detail. $ git branch -r -v origin/HEAD -> origin/master origin/f10/master 0b8a1ce dist-git conversion origin/f11/master 0e4ee2b dist-git conversion origin/f12/master 3031bf9 Merge branch 'f13/master' origin/f13/master 3031bf9 Merge branch 'f13/master' origin/f14/master 3031bf9 Merge branch 'f13/master' origin/f7/master 724adb9 dist-git conversion origin/f8/master 79889bd dist-git conversion origin/f9/master 2a51003 dist-git conversion origin/master 3031bf9 Merge branch 'f13/master' The "master" branch is for rawhide. The fN/master branches are for the primary maintenance for each Fedora release stream. The reason they are called fN/master instead of just fN is that some package maintenance teams might have other branches called fN/something going for new side versions meant for fN being worked on before they are ready to put those in the main fN-updates stream. I don't think that sort of scenario will ever come up for us with systemtap, so just know the branch naming convention but don't worry about it. We have generally maintained Fedora's systemtap packages in lock-step, where we just update each Fedora live stream to each new "upstream" release we make all at the same time, so all the live branches are identical. Heretofore that has been annoyingly informal with the dist-cvs non-branch multi-stream scheme. Now we can just use normal git method to do this, and it will be much, much nicer. But that does require some care and understanding of "normal" git for branch juggling. After the conversion, we had separate identical trees for rawhide and f13, but the branches did not share any history in git's eyes. Likewise, we had an f12 branch that had drifted only very slightly (with one tiny .spec change) but git thought it entirely separate. I've unified the branches with some appropriate git merge commands. That gets the result you see above, where the f1[234]/master branches and master are all actually the very same commit id. Here's what I did: $ fedpkg clone systemtap $ cd systemtap That's just getting started. $ git checkout f12/master Done the first time (when no local f12/master branch exists), this is the same as 'git checkout -b f12/master origin/f12/master', and --track is implicit there by default git settings. That means you get a local f12/master branch that matches the f12/master on the server (i.e. "origin"), and later plain 'git pull' while on this local branch will merge in the server's f12/master branch to yours. Now that this branch exists locally, 'git checkout f12/master' later will simply switch your checkout to that branch. $ git diff f12/master..master This shows that they were not identical trees to begin with. The difference was the 1.3-1 -> 1.3-2 spec change: +%ifarch ppc # crash is not available +%{!?with_crash: %global with_crash 0} +%else %{!?with_crash: %global with_crash 1} +%endif et al. I checked the crash.spec from f13/rawhide and verified that we really do want this change not just for f12, but for later ones too. So here we have an example where in the old system we changed f12 only but really wanted to change f13 and rawhide too, but didn't because the old system made it an unnatural hassle requiring extra thought and legwork. So, we really want the f12/master version of things to be what we have on master and f13/master (and now f14/master, just branched yesterday). $ git checkout master This just switches me back to the local 'master' branch, which already existed by default from 'git clone' (or 'fedpkg clone'). $ git diff master..origin/f13/master This shows that the trees are actually identical for the f13 and rawhide branches already. So there is nothing concrete to worry about, we just want to get git to understand their relationship after the conversion. $ git merge origin/f13/master This merges f13/master into rawhide. The trees are identical already, so this just works automatically. Now git knows that the master and f13/master branches share history, so future merges will work with the normal intelligent behavior for nonidentical trees. $ git merge f12/master Now we're merging the f12/master branch into the master branch. Since git didn't think they shared any history, it shows that one change as a merge conflict, even though a similar case in the future would just merge automatically. I just fixed this up by hand, since that is the safest way to be sure you know what you are getting before you get it. That is, editting systemtap.spec, 'git add -u', 'git commit'. (Here when we know it's what we want, we could have also done 'git merge -s recursive -Xtheirs' and no manual steps required. But that is dangerous magic best left to gurus--I might be a guru, and even I didn't actually use it.) The f11 and earlier branches are actually different (never got 1.3), so I didn't touch those. These branches are effectively dead, since we are not doing package updates for those Fedora releases any more. $ git checkout f12/master $ git merge master Now we switch back to the f12 branch and merge master back into it. This is a no-op on the tree, but it makes git realize the shared history. In fact, it is a "fast-forward" merge, meaning that it just resets the f12/master head to the new master head, because git noticed that the old f12/master head was already fully merged in here. $ git checkout f13/master Now we're getting a local f13/master branch, which we hadn't yet bothered with. (We as well could have done this earlier, and then used f13/master in place of origin/f13/master above.) $ git merge master Now we merge master into f13/master. Just like with f12/master, it notices that we're fully merged with the old f13/master head (thanks to 'git merge origin/f13/master' above), so this is a "fast-forward" too. $ git checkout f14/master $ git merge master Finally, we remember that there is now an f14/master branch (as of today, rawhide is technically targetting f15) and do the same there. Since the old f14/master head was the old master head, it's fully merged up and we get another fast-forward. $ git branch -v f12/master 3031bf9 Merge branch 'f13/master' f13/master 3031bf9 Merge branch 'f13/master' f14/master 3031bf9 Merge branch 'f13/master' * master 3031bf9 Merge branch 'f13/master' Now we're at the state seen above. All the branches are identical. $ git push Now that's on the server too, for all to share. (If you look at the git history, you might see signs that some of the merges actually went in different directions from what I've listed here. I did it all piecemeal before writing this up, and I modified the description to match what I might have done if I'd thought it all through before I did anything. But the material effect of the final state of all the branches is the same.) Now that we've gotten to a good state after the conversion, how will "normal" procedures look in the future? Say, when 1.4 is ready. The first thing to remember is that there might be other changes to the package repository made by Fedora folks. For anything substantive, they should presumably have talked to a systemtap package maintainer so it won't be a surprise. But it's likely there will be things like a bump of the Release: number for a rebuild. So let's imagine this has happened on f14/master, e.g. 2 changed to 3 and a %changelog entry that says "- rebuilt for blah blah". Now f14/master is not the same as master (or f13/master, etc). But we have fresh new stuff to replace it with. What to do? $ git checkout master $ git pull It's reasonable practice to always work on rawhide first. So let's start our "lock-step" procedure with updating rawhide and then making everything else lock into step with that. The 'pull' makes sure you are looking at the current state on the server. Your local branches won't update by themselves. If you want to keep from getting confused, you probably want to do: $ git checkout master && git pull $ git checkout f14/master && git pull $ git checkout f13/master && git pull $ git checkout f12/master && git pull before you start thinking about merges. Now all your local branches match the remote ones, and you can verify that with 'git branch -v -a' to be a little paranoid. $ git checkout master $ git merge f14/master We're merging in those .spec changes that got done on Fedora branches. This is not the only choice for the situation, but IMHO it is the good way to go. It keeps the %changelog history of the old Fedora releases in the new most-current systemtap.spec, which lets those branches both become identical to master again and not lose any of their %changelog history that talks about Fedora builds that exist out in the world. It won't hurt to make that: $ git merge f12/master $ git merge f13/master $ git merge f14/master for all the still-live "lock-step" branches. It's unlikely that any but the most recent one got touched by other Fedora folks, but just in case. Really, it would be the best idea to do this *before* cutting the next systemtap release. Then we can take the merged-up systemtap.spec and fold it back into the systemtap.spec in the systemtap source tree proper. That way we wind up with identical systemtap.spec in Fedora and in our sourceware releases. $ fedpkg new-sources systemtap-1.4.tar.gz $ cp .../systemtap-source/systemtap.spec . That's really only cp if they were made identical before the 1.4 release got made. Edit by hand and verify by eyeball as needed. $ git commit -a -m"Update to 1.4" Or however you like to do it, get the changes committed. Before this you presumably did some build tests of some sort. Or you did: $ fedpkg local which does a local rpmbuild of the srpm made from what's in your working directory, so you can test it out beforehand. $ git push $ fedpkg build This runs koji for you, so you can see the rawhide build succeed before moving on to other branches. Of course, if you are confident about things, you don't need to wait for that build to finish successfully before doing other git work. $ git checkout f12/master $ git merge master $ git push $ fedpkg build This returns f12/master to lock-step (same commit id) as master, and builds it too. Repeat for f13, f14. (Or do f14, then f13, then f12, or whatever floats your boat.) Note that 'fedpkg build' looks at which origin/* branch your current local branch is tracking, and uses the fN/* naming convention to decide which build target to point your koji build at. So actually the name of the local branch doesn't matter (if you want to do fancier things in your local git repo), but the local branch has to be "tracking" (i.e. 'git config branch.<branchname>.merge' setting) the right Fedora server branch. Using 'git checkout --track -b mybranch origin/somebranch' is the usual way to get that set up right from the start when you make a new branch. I hope this helps make the next rounds of systemtap releases and associated Fedora shenanigans go smoothly and pleasantly. I am really quite pleased with the improvements of the git-based system over the old CVS-with-insane-convention-choices system, but all of that new goodness does essentially come from being able to do these arcane things with git. So I imagine it's all still rather daunting to anyone who has less experience with juggling branches in git. As git goes, what we want to do here is really quite simple, but that leaves it at pretty subtle and nonobvious in the absolute sense. Thanks, Roland