MarGolf Meets Langton's Anthill
Java 10, 611 609 607 593 591 bytes

m-&gt;{int x=0,y=0,X=0,Y=0,r=10,c,d=4,e;for(d*=Math.random();r--&gt;0;)for(c=20;c--&gt;0;){if(m[r][c]&lt;43){x=r;y=c;}if(m[r][c]&gt;48){X=r;Y=c;}}for(;;d=m[r][c]&lt;45?d&lt;2?d+2:d&lt;3?1:0:d&gt;1?d-2:d&gt;0?2:3){p(m);for(e=4;e==4;e*=Math.random())e=9;m[r=x][c=y]=m[x+=e/3&lt;1?x&gt;0?-1:1:e/3&gt;1?x&lt;9?1:-1:0][y+=e%3&lt;1?y&gt;0?-1:1:e%3&gt;1?y&lt;19?1:-1:0];if(m[x][y]&gt;48){m[x][y]=48;m[r][c]=0;p(m);e/=0;}m[x][y]=42;m[r=X][c=Y]=m[X+=d&lt;1?X&lt;9?1:-1:d==1?X&gt;0?-1:1:0][Y+=d==2?Y&lt;19?1:-1:d&gt;2?Y&gt;0?-1:1:0];if(m[X][Y]&lt;43){m[r][c]=48;m[X][Y]=0;p(m);e/=0;}m[X][Y]=79;}}void p(char[][]m){var p=&quot;&quot;;for(var a:m)p+=p.valueOf(a)+&quot;\n&quot;;System.out.println(p);}

-4 bytes thanks to @ceilingcat.
Assumes the final swap of * and O will empty the cell where * is coming from.
Explanation:
Try it online.
m-&gt;{                        // Method with character-matrix parameter and no return-type
  int x=0,y=0,              //  [x,y] coordinates of MarGolf
      X=0,Y=0,              //  [X,Y] coordinates of Langton's Anthill
      r=10,c,               //  Temp [x,y] coordinates
      d=4,                  //  Direction Langton's Anthill
      e;                    //  Direction MarGolf
  for(d*=Math.random();     //  Set the direction Langton's Anthill randomly [0,3]
      r--&gt;0;)               //  Loop over the rows:
    for(c=20;c--&gt;0;){       //   Inner loop over the columns:
      if(m[r][c]&lt;43){       //    If '*' is found:
        x=r;y=c;}           //     Set MarGolf's [x,y] coordinates
      if(m[r][c]&gt;48){       //    If 'O' is found:
        X=r;Y=c;}}          //     Set Langton's Anthill's [X,Y] coordinates
  for(;                     //  Loop indefinitely:
       ;                    //    After every iteration:
        d=                  //     Change the direction of Langton's Anthill:
          m[r][c]&lt;45?       //      If the swapped cell contained a comma:
           d&lt;2?d+2:d&lt;3?1:0  //       Change the direction clockwise
          :                 //      Else (the swapped cell contained a dot):
           d&gt;1?d-2:d&gt;0?2:3){//       Change the direction counterclockwise
  p(m);                     //  Pretty-print the matrix
  for(e=4;e==4;e*=Math.random())e=9;
                            //  Change direction MarGolf randomly [0-9] (excluding 4)
  m[r=x][c=y]               //  Save the current MarGolf coordinates
   =m[x+=e/3&lt;1?x&gt;0?-1:1:e/3&gt;1?x&lt;9?1:-1:0]
     [y+=e%3&lt;1?y&gt;0?-1:1:e%3&gt;1?y&lt;19?1:-1:0];
                            //  And change that cell to the content in direction `e`
                            //  0 to 9 (excl. 4) is NW,N,NE,W,n/a,E,SW,S,SE respectively
                            //  If `e` would go out of bounds, it moves opposite instead
  if(m[x][y]&gt;48){           //  If MarGolf reached Langton's Anthill:
    m[x][y]=48;             //   Set that cell to '0'
    m[r][c]=0;              //   And empty the swapped cell
    p(m);                   //   Print the final status of the matrix
    e/=0;}                  //   And stop the loop with an error to exit the program
  m[x][y]=42;               //  Change the cell in the new coordinate to '*'
  m[r=X][c=Y]               //  Save the current Langton's Anthill coordinates
   =m[X+=d&lt;1?X&lt;9?1:-1:d==1?X&gt;0?-1:1:0]
      [Y+=d==2?Y&lt;19?1:-1:d&gt;2?Y&gt;0?-1:1:0];
                            //  And change that cell to the content in direction `d`
                            //  0 to 3 is E,W,S,N respectively
                            //  If `d` would be out of bounds, it moves opposite instead
  if(m[X][Y]&lt;43){           //  If MarGolf reached Langton's Anthill:
    m[r][c]=48;             //   Set that cell to '0'
    m[X][Y]=0;              //   And empty the swapped cell
    p(m);                   //   Print the final status of the matrix
    e/=0;}                  //   And stop the loop with an error to exit the method
  m[X][Y]=79;}}             //  Change the cell in the new coordinate to 'O'

void p(char[][]m){          // Separated method to print the given matrix
  var p=&quot;&quot;;                 //  String to print, starting empty
  for(var a:m){             //  Loop over the rows:
    p+=p.valueOf(a)         //   Convert the character-array to a String line and append,
                     +&quot;\n&quot;; //   including a trailing newline
  System.out.println(p);}   //  Print the String with trailing newline as separator