Recursive Drush make, working with different git branches Setup

For every site we make a custom Drupal install profile is created, copies of which are built using drush make (through Aegir, though that should not make a difference).

There is a top-level build makefile for downloading Drupal core and our install profile, and the install profile contains a makefile specifying Features to download, each of which have a makefile specifying their dependencies (either custom modules in private git repos or contrib modules).

The build makefile (build-SITENAME.make) specifies the install profile like this:

projects[SITENAME_profile][type] = profile
projects[SITENAME_profile][download][type] = git
projects[SITENAME_profile][download][url] = git.hostname:drupal-sites/SITENAME_profile


Then the install profile's makefile will specify a bunch of Features to download, for example:

projects[blog][type] = module
projects[blog][subdir] = features
projects[blog][download][type] = git
projects[blog][download][url] = git.hostname:drupal-features/blog


Which will then download custom and contrib modules and so on.

The above setup has worked well for a couple of years.

Problem

The problem occurs when trying to use git branches for managing different stages of development, e.g. development, staging and production; and having CI automatically manage Drupal sites for each branch. If a build is triggered for the development branch, drush make has to get the correct branch for every one of our repos it downloads.

Given our multi-repo and recursive makefile structure, how would drush make be used to recursively download the correct branch from our git repos?

Things already tried


drush make allows downloading from a git branch, which seemed good
for our use-case. For example, for each makefile (in the build,
install profile and Features repositories) a line is added that
looks like:

projects[blog][download][branch] = development


When the branch of the repo being downloaded matches the branch of the
repo the makefile is in, custom modules specified in the
development branch of the 'blog' Feature will be downloaded from
that module's development branch. Great!

However, this causes problems merging the branches of any repo containing a makefile (given our setup that's quite a lot), we will have to be extremely careful not to accidentally merge [branch] = development into master on any of them.
Checked drush make documentation for a command-line option to specify the default branch for git repository downloads. Found nothing.
Searched the Internet for people with similar quandaries, it seems many are dumping all the dependencies for whole projects into single make files, but that breaks the modularity of Features.


Note: I'm not polling or asking for discussion. I believe there's something silly I'm missing and/or something everyone else is doing that I'm not.
This is a great question! I maintain both Aegir and Drush Make, but I'm not certain that I have a really good answer for you.


I think that you're suggestion in (1) is probably the best bet at
the moment. It has the virtue of already working, as you've
mentioned. The issue with accidentally merging these [branch]
attributes is still valid though. The solution though, lies not in
Drush Make, but in Git!

Check out this article. It describes using Git attributes,
along with a custom (null) Git merge driver. To summarize, each
developer would have to run the following:

git config --global merge.ours.driver true


This basically means that when an item is flagged to use the
'ours' merge driver, it'll just call /bin/true, hence my earlier
reference to it being a null operation.

But, how do you flag your makefiles to use this driver? That's where
Git attributes come into play. Create a .gitattributes file at the root of your repo (i.e., along side .gitignore). Then add the
following line:

build-SITENAME.make merge=ours


You'll probably want to do the same for your feature repos as well.
As for solutions leveraging Drush Make, I don't know that there's a
solid solution.

A few years back, I experimented with using overrides trying to do
something similar, as described in the Drush Make
documentation. The basic idea is that you can change (override)
attributes from included makefiles.

However, in the case of recursive makefiles, they are built
separately, instead of being merged into the parent makefile prior
to build. As a result, there's no opportunity to override their
attributes. This may have changed in the time since I tested it, but
I think the underlying mechanics remain largely the same, and so
it's unlikely to work.

Setting a default in a parent makefile would be a nice way to
accomplish something like this too. I haven't tested this, but
again, I'm pretty sure the default wouldn't apply to the recursively
built makefile.

Instead, an option would be to build your parent makefile without
recursion, but rather including the feature makefiles directly. You
can now include makefiles directly from within a Git repo
(using the new YAML syntax):

includes:
  # A file from a git repository.
  makefile: "example_dir/example.make"
  download:
    type: "git"
    url: "git@github.com:organisation/repository.git"
    branch: "master"


You'd either want to call drush make with the --no-recursion
option, or specify it in your makefiles:

--no-recursion                   Do not recurse into the makefiles of any
                                 downloaded projects; you can also set 
                                 [do_recursion] = 0 on a
                                 per-project basis in the makefile.



I'm sure there are other options available, but those are the ones that I think would least impact your existing workflows. I'd be very curious to see other suggestions.

There is a feature request to allow the use of variables in .make files, which might address such a use-case more directly, if it were implemented. Perhaps we could look for such variables in options passed on the command line, or in environment variables.

That said, in the long run, I believe that Drush Make will be supplanted entirely by Composer, so it might be more fruitful to explore in that direction, if the suggestions above don't work out.