How to "UPDATE table SET col = (CASE WHEN ... THEN (SELECT col FROM table ORDER BY col DESC LIMIT 1) END)"?
I'm not sure of your reasoning for wanting to do it within a single query, but the one thing I would ask though is that you think about the maintenance of such a query - you may not be the only person who has to support this.
It took me a while to understand what your query was doing, and in the end I ignored it completely and continued based upon the question you were asking. It would have been much simpler, easier to understand and support if you used a single statement for each step and then wrap it in a transaction for consistency. Depending on the number of rows in the table, it might even be more performant as you won't need to join the table to itself a few times.
However the below will achieve what you want:
CREATE TABLE list (
    id int unsigned NOT NULL AUTO_INCREMENT,
    parent_id int unsigned,
    position int unsigned,
    PRIMARY KEY(id)
);

-- parents
INSERT INTO list (position) VALUES (0),(1);
-- parent 1 children
INSERT INTO list (parent_id, position) VALUES
  (1,0), (1,1), (1,2), (1,3), (1,4), (1,5);
-- parent 2 children
INSERT INTO list (parent_id, position) VALUES
  (2,0), (2,1), (2,2), (2,3), (2,4), (2,5);

WITH a AS 
(
    SELECT parent_id, MAX(position) as max_position
      FROM list
  GROUP BY parent_id
)
   UPDATE list AS b
     JOIN a
       ON a.parent_id = b.parent_id
LEFT JOIN list AS c
       ON c.parent_id = b.parent_id
      AND c.position &gt; b.position
      SET b.parent_id = 1
        , b.position = a.max_position + 1
        , c.position = c.position - 1
    WHERE b.id = 11;
 
SELECT * 
  FROM list 
 ORDER BY parent_id, position;

Below are the results when moving a item that is not the last child of a parent.
Before

After

Below are the results when moving a item that is the last child of a parent.
Before

After