What's the difference between Adaptive Control and Hierarchical Reinforcement Learning?
The difference between the two tasks really comes down to the level of continuity assumed in the models of the problem.

In adaptive control, continuity is assumed at all levels; the problem space and the actions to be executed are all continuous.

In hierarchical reinforcement learning, although the problem space is continuous, the actions to execute upon the space are discrete.

A good analogy for this is the robot arm trying to move to a certain goal. Adaptive control would work with Jacobian to find a continuous solution to this problem. Reinforcement learning would select actions (up, down, left, right) to learn the problem space and find a solution. 

Hierarchical reinforcement learning and adaptive control are really complements of each other. When the problem gets too complex to compute with Adaptive Control, hierarchical learning steps in to make it discrete and more scalable.

For another perspective, especially since my explanation breaks down when you consider continuous HRL (which does exist), consider this explanation from my personal correspondence with Daniel Rasumussan:


  Practically, there are many differences between how they [Adaptive Control and HRL in a continuous space] are
  implemented; they use different algorithms, different learning rules,
  different representations, etc.  For example, adaptive control is not
  typically hierarchical.  You could arguably call it a hierarchy with a
  depth of 1 (with e.g. the "high level" being control in the x,y space
  and "low level" being control in the motor space).  But there's no
  temporal abstraction in that hierarchy; the high level isn't trying to
  encapsulate low level decisions, the problem is how to find a nice
  reliable one-to-one mapping between those two spaces.  Of course you
  could make adaptive control hierarchical, as you can make pretty much
  anything hierarchical, but that isn't typically what they're
  interested in.  Perhaps a more critical difference is that in motor
  control typically it is assumed that the transition and reward
  functions are known to the agent.  There may be some unknown
  variability in the case of adaptive control, but by and large the
  agent has a pretty good idea what the effects of its actions will be,
  and what the goal is that it is trying to achieve.  It's one of the
  key principle of reinforcement learning that that information is not
  known, and must be figured out by the agent by exploring the
  environment.  Now again here, you could come up with a motor control
  problem where the agent doesn't assume that information is known, but
  that just isn't the kind of problem those techniques are designed for,
  in general.  So, to sum up, there certainly is a gradient between HRL
  and adaptive control, and by tweaking various assumptions you can
  bring them closer together or farther apart.  But just implementing
  continuous actions in HRL would still leave you quite a ways from
  adaptive control.


As for an example combination this, one might imagine using hierarchical reinforcement learning to play a video game like Pong and then changing the physics of the game (Pong on slippery ice!) so that Adaptive Control might step in.