Disable DNS assignment with Airport Extreme IPv6 tunnel
That's a sneaky one - I'm sure you looked into the Airport Utility configuration.
Tried to assign a static IPv6 to it? Fiddle with IPv6 DNS&amp; share setting. 

I don't know if there is a way to turn off the RA - since they are supposed to "maintain" network quality, and this is a connect&amp;forget device so maybe that's a default and it's fixed. I dont know a way to send ipv6 nd ra suppress all to it :-)

If you assign a static IPv6 to a rr, it still uses the info from the RA's in the ICMPv6 (type 134) that comes along to get the default gateway and/or the mac+link local from the first hop. See what happens when you fill static IPv6 where possible.